Societal biases that are contained in retrieved documents have received increased interest. Such biases, which are often prevalent in the training data and learned by the model, can cause societal harms, by misrepresenting certain groups, and by enforcing stereotypes. Mitigating such biases demands algorithms that balance the trade-off between maximized utility for the user with fairness objectives, which incentivize unbiased rankings. Prior work on bias mitigation often assumes that ranking scores, which correspond to the utility that a document holds for a user, can be accurately determined. In reality, there is always a degree of uncertainty in the estimate of expected document utility. This uncertainty can be approximated by viewing ranking models through a Bayesian perspective, where the standard deterministic score becomes a distribution. In this work, we investigate whether uncertainty estimates can be used to decrease the amount of bias in the ranked results, while minimizing loss in measured utility. We introduce a simple method that uses the uncertainty of the ranking scores for an uncertainty-aware, post hoc approach to bias mitigation. We compare our proposed method with existing baselines for bias mitigation with respect to the utility-fairness trade-off, the controllability of methods, and computational costs. We show that an uncertainty-based approach can provide an intuitive and flexible trade-off that outperforms all baselines without additional training requirements, allowing for the post hoc use of this approach on top of arbitrary retrieval models.

翻译：检索文档中包含的社会偏见已引起越来越多的关注。这些偏见通常普遍存在于训练数据中并被模型学习，可能通过歪曲特定群体和强化刻板印象造成社会危害。缓解此类偏差需要算法在用户效用最大化与公平目标（激励无偏排序）之间取得平衡。既往关于偏差缓解的研究通常假设排序得分（即文档对用户的效用）能够被精确确定。实际上，对于文档期望效用的估计总是存在一定程度的不确定性。通过从贝叶斯视角审视排序模型（其中标准确定性得分变为分布），可以近似这种不确定性。本研究探讨不确定性估计能否在最小化已测效用损失的同时，减少排序结果中的偏差量。我们提出一种简单方法，利用排序得分的不确定性，以不确定性感知的后处理方法进行偏差缓解。我们将所提方法与现有基线方法进行对比，评估维度包括效用-公平权衡、方法可控性及计算成本。研究表明，基于不确定性的方法能够提供直观且灵活的权衡，在无需额外训练需求的前提下优于所有基线方法，且可针对任意检索模型进行后处理应用。