The Fairness of Ranking Procedure in Pair-wise Preference Learning

Abstract

In pair-wise preference learning, a crucial point is how to decode the predictions of the pair-wise preference to a final preference order - a ranking procedure. Simple voting, iterated choice, and Slater-optimal ranking are usual techniques, but their ranking results are usually very different from each other. Hitherto, experimentation is the main method of estimating the ranking approaches, and the formal estimation is still an open question. The main contribution of this paper is the definition of a framework to import the fairness theory of preference aggregation to estimate the ranking procedure, where every pair-wise preference learner is seen as an agent and the ranking procedure is seen as a special case of multiple agents' preferences aggregation. In addition, by transformed into a special aggregation case of RANK voting rule, if there are at least three labels, then simple voting and iterated choice are proved to be not fair for their dependence to irrelevant alternatives.