Mitigating the disparate impact of statistical machine learning methods is crucial for ensuring fairness. While extensive research aims to reduce disparity, the effect of using a \emph{finite dataset} -- as opposed to the entire population -- remains unclear. This paper explores the statistical foundations of fair binary classification with two protected groups, focusing on controlling demographic disparity, defined as the difference in acceptance rates between the groups. Although fairness may come at the cost of accuracy even with infinite data, we show that using a finite sample incurs additional costs due to the need to estimate group-specific acceptance thresholds. We study the minimax optimal classification error while constraining demographic disparity to a user-specified threshold. To quantify the impact of fairness constraints, we introduce a novel measure called \emph{fairness-aware excess risk} and derive a minimax lower bound on this measure that all classifiers must satisfy. Furthermore, we propose FairBayes-DDP+, a group-wise thresholding method with an offset that we show attains the minimax lower bound. Our lower bound proofs involve several innovations. Experiments support that FairBayes-DDP+ controls disparity at the user-specified level, while being faster and having a more favorable fairness-accuracy tradeoff than several baselines.

翻译：减轻统计机器学习方法的差异化影响对于确保公平性至关重要。虽然大量研究旨在减少差异，但使用有限数据集（而非整个人群）的影响仍不明确。本文探讨了具有两个受保护群体的公平二分类的统计基础，重点关注控制人口统计学差异（定义为群体间接受率之差）。尽管即使使用无限数据，公平性也可能以准确性为代价，但我们表明，使用有限样本会因需要估计群体特定的接受阈值而产生额外成本。我们在将人口统计学差异限制在用户指定阈值的同时，研究最小最大最优分类误差。为了量化公平性约束的影响，我们引入了一种称为“公平感知超额风险”的新度量，并推导了所有分类器必须满足的该度量的最小最大下界。此外，我们提出了FairBayes-DDP+，一种带有偏移量的逐组阈值方法，我们证明该方法达到了最小最大下界。我们的下界证明涉及多项创新。实验支持FairBayes-DDP+在用户指定水平上控制差异，同时其速度更快，且公平性-准确性权衡优于多个基线方法。