The ``impossibility theorem'' -- which is considered foundational in algorithmic fairness literature -- asserts that there must be trade-offs between common notions of fairness and performance when fitting statistical models, except in two special cases: when the prevalence of the outcome being predicted is equal across groups, or when a perfectly accurate predictor is used. However, theory does not always translate to practice. In this work, we challenge the implications of the impossibility theorem in practical settings. First, we show analytically that, by slightly relaxing the impossibility theorem (to accommodate a \textit{practitioner's} perspective of fairness), it becomes possible to identify a large set of models that satisfy seemingly incompatible fairness constraints. Second, we demonstrate the existence of these models through extensive experiments on five real-world datasets. We conclude by offering tools and guidance for practitioners to understand when -- and to what degree -- fairness along multiple criteria can be achieved. For example, if one allows only a small margin-of-error between metrics, there exists a large set of models simultaneously satisfying \emph{False Negative Rate Parity}, \emph{False Positive Rate Parity}, and \emph{Positive Predictive Value Parity}, even when there is a moderate prevalence difference between groups. This work has an important implication for the community: achieving fairness along multiple metrics for multiple groups (and their intersections) is much more possible than was previously believed.
翻译:“不可能定理”——在算法公平性文献中被视为基础性结论——断言,在拟合统计模型时,常见的公平性概念与性能之间必然存在权衡,除非出现两种特殊情况:预测结果的发生率在各群体间相等,或者使用了完全准确的预测器。然而,理论并不总能转化为实践。在这项工作中,我们挑战了不可能定理在实际场景中的含义。首先,通过分析表明,通过略微放宽不可能定理(以适应从业者对公平的视角),可以识别出一大类满足看似不相容的公平约束的模型。其次,我们通过五个真实数据集的广泛实验证明了这些模型的存在性。最后,我们提供工具和指导,帮助从业者理解何时——以及在何种程度上——能够实现多重公平标准。例如,如果允许指标之间存在较小的误差边际,那么存在一大类模型同时满足假阴性率均等、假阳性率均等和阳性预测值均等,即使在群体间的发生率存在中等差异的情况下也是如此。这项工作对学术界具有重要意义:实现多个群体(及其交叉群体)在多个指标上的公平性远比之前认为的更加可行。