The presence of inequity is a fundamental problem in the outcomes of decision-making systems, especially when human lives are at stake. Yet, estimating notions of unfairness or inequity is difficult, particularly if they rely on hard-to-measure concepts such as risk. Such measurements of risk can be accurately obtained when no unobserved confounders have jointly influenced past decisions and outcomes. However, in the real world, this assumption rarely holds. In this paper, we show a surprising result that one can still give meaningful bounds on treatment rates to high-risk individuals, even when entirely eliminating or relaxing the assumption that all relevant risk factors are observed. We use the fact that in many real-world settings (e.g., the release of a new treatment) we have data from prior to any allocation to derive unbiased estimates of risk. This result is of immediate practical interest: we can audit unfair outcomes of existing decision-making systems in a principled manner. For instance, in a real-world study of Paxlovid allocation, our framework provably identifies that observed racial inequity cannot be explained by unobserved confounders of the same strength as important observed covariates.

翻译：不公平性是决策系统结果中的一个根本性问题，尤其当涉及人类生命时更为突出。然而，评估不公平或不平等概念十分困难，特别是当这些概念依赖于风险等难以测量的指标时。当没有未观测混杂因素共同影响历史决策和结果时，可以准确获取此类风险测量值。但在现实世界中，这一假设很少成立。本文展示了一个令人惊讶的结果：即使完全消除或放宽"所有相关风险因素均被观测"的假设，我们仍然能够对高风险个体的处置率给出有意义的界。我们利用了在诸多现实场景（如新疗法的发布）中可获得分配前数据的特性，从而推导出无偏的风险估计。这一结果具有直接的实践意义：我们能够以原则性的方式审计现有决策系统中的不公平结果。例如，在一项关于帕克洛维德（Paxlovid）分配的实地研究中，我们的框架可证明地识别出：观察到的种族不平等无法通过与重要观测协变量强度相当的未观测混杂因素来解释。