Ranking algorithms as an essential component of retrieval systems have been constantly improved in previous studies, especially regarding relevance-based utilities. In recent years, more and more research attempts have been proposed regarding fairness in rankings due to increasing concerns about potential discrimination and the issue of echo chamber. These attempts include traditional score-based methods that allocate exposure resources to different groups using pre-defined scoring functions or selection strategies and learning-based methods that learn the scoring functions based on data samples. Learning-based models are more flexible and achieve better performance than traditional methods. However, most of the learning-based models were trained and tested on outdated datasets where fairness labels are barely available. State-of-art models utilize relevance-based utility scores as a substitute for the fairness labels to train their fairness-aware loss, where plugging in the substitution does not guarantee the minimum loss. This inconsistency challenges the model's accuracy and performance, especially when learning is achieved by gradient descent. Hence, we propose a distribution-based fair learning framework (DLF) that does not require labels by replacing the unavailable fairness labels with target fairness exposure distributions. Experimental studies on TREC fair ranking track dataset confirm that our proposed framework achieves better fairness performance while maintaining better control over the fairness-relevance trade-off than state-of-art fair ranking frameworks.
翻译:排序算法作为检索系统的核心组件,在以往研究中不断得到改进,尤其是在基于相关性的效用方面。近年来,由于对潜在歧视和回声室问题的日益关注,越来越多的研究尝试关注排序中的公平性。这些尝试包括传统的基于分数的方法(通过预定义的评分函数或选择策略将曝光资源分配给不同群体)以及基于学习的方法(基于数据样本学习评分函数)。基于学习的模型比传统方法更灵活且能获得更好的性能。然而,大多数基于学习的模型是在公平性标签几乎不可用的过时数据集上进行训练和测试的。最先进的模型利用基于相关性的效用分数作为公平性标签的替代来训练其公平性感知损失函数,但这种替代并不能保证损失最小化。这种不一致性挑战了模型的准确性和性能,特别是在通过梯度下降进行学习时。因此,我们提出了一种基于分布的公平学习框架(DLF),该框架通过用目标公平性曝光分布替代不可用的公平性标签,从而无需标签。在TREC公平排序赛道数据集上的实验研究证实,与最先进的公平排序框架相比,我们提出的框架在保持对公平性与相关性权衡更好控制的同时,实现了更优的公平性性能。