Many definitions of fairness or inequity involve unobservable causal quantities that cannot be directly estimated without strong assumptions. For instance, it is particularly difficult to estimate notions of fairness that rely on hard-to-measure concepts such as risk (e.g., quantifying whether patients at the same risk level have equal probability of treatment, regardless of group membership). Such measurements of risk can be accurately obtained when no unobserved confounders have jointly influenced past decisions and outcomes. However, in the real world, this assumption rarely holds. In this paper, we show that, surprisingly, one can still compute meaningful bounds on treatment rates for high-risk individuals (i.e., conditional on their true, \textit{unobserved} negative outcome), even when entirely eliminating or relaxing the assumption that we observe all relevant risk factors used by decision makers. We use the fact that in many real-world settings (e.g., the release of a new treatment) we have data from prior to any allocation to derive unbiased estimates of risk. This result enables us to audit unfair outcomes of existing decision-making systems in a principled manner. We demonstrate the effectiveness of our framework with a real-world study of Paxlovid allocation, provably identifying that observed racial inequity cannot be explained by unobserved confounders of the same strength as important observed covariates.

翻译：许多公平性或不平等的定义涉及无法直接估计的不可观测因果量，除非施加强假设。例如，依赖难以衡量的概念（如风险）的公平性度量尤其难以估计（例如，量化处于相同风险水平的患者是否具有平等的治疗概率，无论其群体归属如何）。当不存在未观测混杂因素共同影响过去的决策和结果时，此类风险测量可以准确获得。然而在现实世界中，这一假设很少成立。本文证明，令人惊讶的是，即使完全消除或放宽“我们观测到决策者使用的所有相关风险因素”这一假设，我们仍能计算高风险个体（即以其真实、未观测的负面结果为条件）治疗率的有意义边界。我们利用以下事实：在许多现实场景中（例如新疗法的发布），我们拥有任何分配实施前的数据，从而推导出风险的无偏估计。这一结果使我们能够以原则性的方式审计现有决策系统的不公平结果。我们通过对Paxlovid分配的真实世界研究证明本框架的有效性，可验证地识别出：观测到的种族不平等无法用与重要观测协变量同等强度的未观测混杂因素来解释。