The pursuit of fairness in machine learning (ML), ensuring that the models do not exhibit biases toward protected demographic groups, typically results in a compromise scenario. This compromise can be explained by a Pareto frontier where given certain resources (e.g., data), reducing the fairness violations often comes at the cost of lowering the model accuracy. In this work, we aim to train models that mitigate group fairness disparity without causing harm to model accuracy. Intuitively, acquiring more data is a natural and promising approach to achieve this goal by reaching a better Pareto frontier of the fairness-accuracy tradeoff. The current data acquisition methods, such as fair active learning approaches, typically require annotating sensitive attributes. However, these sensitive attribute annotations should be protected due to privacy and safety concerns. In this paper, we propose a tractable active data sampling algorithm that does not rely on training group annotations, instead only requiring group annotations on a small validation set. Specifically, the algorithm first scores each new example by its influence on fairness and accuracy evaluated on the validation dataset, and then selects a certain number of examples for training. We theoretically analyze how acquiring more data can improve fairness without causing harm, and validate the possibility of our sampling approach in the context of risk disparity. We also provide the upper bound of generalization error and risk disparity as well as the corresponding connections. Extensive experiments on real-world data demonstrate the effectiveness of our proposed algorithm. Our code is available at https://github.com/UCSC-REAL/FairnessWithoutHarm.

翻译：在机器学习（ML）中追求公平性，即确保模型不对受保护的人口群体表现出偏见，通常会导致一种权衡情境。这种权衡可以通过帕累托前沿来解释：在给定特定资源（例如数据）的情况下，减少公平性违规往往以降低模型准确性为代价。在本研究中，我们的目标是训练能够减轻群体公平性差异且不损害模型准确性的模型。直观上，通过获取更多数据以达到更优的公平性-准确性权衡帕累托前沿，是实现这一目标的一种自然而有效的方法。当前的数据获取方法，例如公平主动学习方法，通常需要标注敏感属性。然而，出于隐私和安全考虑，这些敏感属性标注应当受到保护。本文提出了一种可行的主动数据采样算法，该算法不依赖于训练集中的群体标注，而仅需在小型验证集上进行群体标注。具体而言，该算法首先根据每个新样本在验证数据集上对公平性和准确性的影响进行评分，然后选择一定数量的样本用于训练。我们从理论上分析了获取更多数据如何能在不造成损害的情况下改善公平性，并在风险差异的背景下验证了我们采样方法的可行性。我们还提供了泛化误差与风险差异的上界及其相应关联。在真实世界数据上进行的大量实验证明了我们提出算法的有效性。我们的代码可在 https://github.com/UCSC-REAL/FairnessWithoutHarm 获取。