An hybrid preference learning framework to refine the consensus ranking

The study of preference rankings, or preference learning, is becoming increasingly important in many scientific fields. Preferences are expressed when a group of judges (or raters) evaluate a collection of elements (or items), assigning an order to objects based on which ones are preferred over others. When there are many judges and also a large number of items to be evaluated, an aggregate measure is needed in order to solve the rank aggregation problem, providing an interpretable comparison of the ranked items and assessing the overall level of agreement among judges. The rank aggregation problem is an NP-hard problem because it becomes more difficult as the number of items increases significantly. Approaches such as the branch-and-bound can be applied to problems with a limited number of items (i.e. fewer than 200). When the number of items grows, heuristic techniques have been developed to provide approximate solutions. Many of these heuristics are based on Kemeny’s axiomatic approach, which has shown to be valid with tied rankings. In this paper, we propose a framework that aims at providing more than just a choice between “this slow but extremely accurate algorithm” and “this just good and faster one”. Thus, following a hybrid approach, the proposal permits a trade-off between the two options. A simulation study shows the performance of the proposed framework in a controlled environment. Furthermore, a real world data set with a large number of items is considered. As a result, the proposal provides significant improvements in the solution found with a reasonable additional amount of computational time. This improvement is mostly investigated while using the recently proposed PSOPR algorithm (as the faster one) and the state of the art QUICK (as the slowest one).