Making fair decisions is crucial to ethically implementing machine learning algorithms in social settings. In this work, we consider the celebrated definition of counterfactual fairness [Kusner et al., NeurIPS, 2017]. We begin by showing that an algorithm which satisfies counterfactual fairness also satisfies demographic parity, a far simpler fairness constraint. Similarly, we show that all algorithms satisfying demographic parity can be trivially modified to satisfy counterfactual fairness. Together, our results indicate that counterfactual fairness is basically equivalent to demographic parity, which has important implications for the growing body of work on counterfactual fairness. We then validate our theoretical findings empirically, analyzing three existing algorithms for counterfactual fairness against three simple benchmarks. We find that two simple benchmark algorithms outperform all three existing algorithms -- in terms of fairness, accuracy, and efficiency -- on several data sets. Our analysis leads us to formalize a concrete fairness goal: to preserve the order of individuals within protected groups. We believe transparency around the ordering of individuals within protected groups makes fair algorithms more trustworthy. By design, the two simple benchmark algorithms satisfy this goal while the existing algorithms for counterfactual fairness do not.

翻译：在社交场景中实现伦理化的机器学习算法，关键在于做出公平决策。本研究探讨了著名的反事实公平性定义[Kusner等, NeurIPS, 2017]。我们首先证明，满足反事实公平性的算法同样满足人口统计均等——这一远为简单的公平性约束。类似地，我们指出所有满足人口统计均等的算法均可通过简单修改达到反事实公平性标准。综合来看，研究结果表明反事实公平性本质上与人口统计均等等价，这对日益壮大的反事实公平性研究领域具有重要启示。随后我们通过实证验证理论发现，将三种现有反事实公平性算法与三个简单基准模型进行对比分析。结果表明，在多个数据集上，两种简单基准算法在公平性、准确率和效率方面均优于所有现有算法。基于分析结果，我们正式提出具体的公平性目标：保持受保护群体内个体的排序顺序。我们认为，明确受保护群体内个体排序的透明度能使公平算法更具可信度。通过设计，两种简单基准算法能满足该目标，而现有反事实公平性算法则无法实现。