Most fair machine learning methods either highly rely on the sensitive information of the training samples or require a large modification on the target models, which hinders their practical application. To address this issue, we propose a two-stage training algorithm named FAIRIF. It minimizes the loss over the reweighted data set (second stage) where the sample weights are computed to balance the model performance across different demographic groups (first stage). FAIRIF can be applied on a wide range of models trained by stochastic gradient descent without changing the model, while only requiring group annotations on a small validation set to compute sample weights. Theoretically, we show that, in the classification setting, three notions of disparity among different groups can be mitigated by training with the weights. Experiments on synthetic data sets demonstrate that FAIRIF yields models with better fairness-utility trade-offs against various types of bias; and on real-world data sets, we show the effectiveness and scalability of FAIRIF. Moreover, as evidenced by the experiments with pretrained models, FAIRIF is able to alleviate the unfairness issue of pretrained models without hurting their performance.

翻译：大多数公平机器学习方法要么高度依赖训练样本的敏感信息，要么需要对目标模型进行大量修改，这阻碍了其实际应用。为解决这一问题，我们提出了一种名为FAIRIF的两阶段训练算法。该算法在第二阶段通过重加权数据集最小化损失，其中样本权重在第一阶段被计算以平衡不同人口统计群体间的模型性能。FAIRIF可广泛应用于通过随机梯度下降训练的各类模型，且无需修改模型结构，仅需在小规模验证集上标注群体属性即可计算样本权重。理论上我们证明，在分类任务中，通过该权重进行训练可缓解三类群体差异度量。合成数据集实验表明，FAIRIF能针对多种偏差类型生成具有更优公平性-效用权衡的模型；在真实数据集上，我们验证了FAIRIF的有效性与可扩展性。此外，预训练模型实验结果证明，FAIRIF能在不损害性能的前提下缓解预训练模型的不公平性问题。