Rankings are ubiquitous across many applications, from search engines to hiring committees. In practice, many rankings are derived from the output of predictors. However, when predictors trained for classification tasks have intrinsic uncertainty, it is not obvious how this uncertainty should be represented in the derived rankings. Our work considers ranking functions: maps from individual predictions for a classification task to distributions over rankings. We focus on two aspects of ranking functions: stability to perturbations in predictions and fairness towards both individuals and subgroups. Not only is stability an important requirement for its own sake, but -- as we show -- it composes harmoniously with individual fairness in the sense of Dwork et al. (2012). While deterministic ranking functions cannot be stable aside from trivial scenarios, we show that the recently proposed uncertainty aware (UA) ranking functions of Singh et al. (2021) are stable. Our main result is that UA rankings also achieve multigroup fairness through successful composition with multiaccurate or multicalibrated predictors. Our work demonstrates that UA rankings naturally interpolate between group and individual level fairness guarantees, while simultaneously satisfying stability guarantees important whenever machine-learned predictions are used.

翻译：在许多应用场景中，从搜索引擎到招聘委员会，排序无处不在。实际上，许多排序源于预测器的输出。然而，当为分类任务训练的预测器存在内在不确定性时，这种不确定性应如何在衍生排序中体现尚不明确。本研究考虑排序函数：将分类任务的个体预测映射到排序分布的函数。我们聚焦于排序函数的两个维度：对预测扰动的稳定性，以及面向个体与子群体的公平性。稳定性不仅是自身的重要需求，更与我们证明的那样——能与Dwork等人(2012)提出的个体公平性完美融合。虽然除平凡场景外，确定性排序函数无法具备稳定性，但Singh等人(2021)近期提出的不确定性感知(UA)排序函数具有稳定性。我们的主要贡献在于：UA排序通过与多精确或多校准预测器的成功组合，能够实现多群体公平性。本研究证明，UA排序能在群体与个体层面公平性保证之间自然插值，同时在使用机器学习预测时满足重要的稳定性保证。