Practical perspectives of consumer fairness in recommendation

In recent years, there has been an increasing number of mitigation procedures against consumer unfairness in personalized rankings. However, the experimental protocols adopted so far for evaluating a mitigation procedure were often fundamentally different (e.g., with respect to the fairness definitions, data sets, data splits, and evaluation metrics) and limited to a narrow set of perspectives (e.g., focusing on a single demographic attribute and/or not reporting any analysis on efficiency). This situation makes it challenging for scientists to consciously decide which mitigation procedure better suits their practical setting. In this paper, we investigated the properties a given mitigation procedure against consumer unfairness should be evaluated on, to provide a more holistic view on its effectiveness. We first identified eight technical properties and evaluated the extent to which existing mitigation procedures against consumer unfairness met these properties, qualitatively and quantitatively (when possible), on two public data sets. Then, we outlined the main trends and open issues emerged from our multi-dimensional analysis and provided key practical recommendations for future research.