An hybrid preference learning framework to refine the consensus ranking

The study of preference rankings (p. learning) is becoming increasingly important in many scientific fields. Preferences are expressed when a group of judges (raters) evaluate a collection of elements (items), assigning an order to objects based on which ones are preferred over others. When there are many judges and also a large number of items to be evaluated, an aggregate measure is needed in order to solve the rank aggregation problem, providing an interpretable comparison of the ranked items and assessing the overall level of agreement among judges. The rank aggregation problem is NP-hard because it becomes more difficult as the number of items increases significantly. Approaches such as the branch-and-bound can be applied to problems with a limited number of items (i.e. $$<200$$<200). When the number of items grows, heuristic techniques have been developed to provide approximate solutions. Many of these heuristics are based on Kemeny’s axiomatic approach, which has shown to be valid with tied rankings. In this paper, we propose a framework that aims at providing more than just a choice between a slow but extremely accurate algorithm and a faster but not so accurate one. Thus, following a hybrid approach, a trade-off between the two options is permitted. A simulation study shows the performance of the proposal in a controlled environment. Furthermore, a real world data set with a large number of items is considered. Results show significant improvements in the solution found, with a reasonable additional amount of computational time. This improvement is mainly investigated whilst using the recently proposed PSOPR algorithm (as the faster one) and the state of the art QUICK (as the slowest one).