Arbitrary, inconsistent, or faulty decision-making raises serious concerns, and preventing unfair models is an increasingly important challenge in Machine Learning. Data often reflect past discriminatory behavior, and models trained on such data may reflect bias on sensitive attributes, such as gender, race, or age. One approach to developing fair models is to preprocess the training data to remove the underlying biases while preserving the relevant information, for example, by correcting biased labels. While multiple label noise correction methods are available, the information about their behavior in identifying discrimination is very limited. In this work, we develop an empirical methodology to systematically evaluate the effectiveness of label noise correction techniques in ensuring the fairness of models trained on biased datasets. Our methodology involves manipulating the amount of label noise and can be used with fairness benchmarks but also with standard ML datasets. We apply the methodology to analyze six label noise correction methods according to several fairness metrics on standard OpenML datasets. Our results suggest that the Hybrid Label Noise Correction method achieves the best trade-off between predictive performance and fairness. Clustering-Based Correction can reduce discrimination the most, however, at the cost of lower predictive performance.

翻译：任意、不一致或有缺陷的决策引发了严重担忧，而防止模型不公平成为机器学习中日益重要的挑战。数据往往反映了过去的歧视性行为，基于此类数据训练的模型可能对性别、种族或年龄等敏感属性表现出偏差。开发公平模型的一种方法是通过预处理训练数据来消除潜在偏差，同时保留相关信息，例如通过校正有偏标签。尽管存在多种标签噪声校正方法，但关于其在识别歧视行为方面表现的信息非常有限。本研究开发了一种实证方法论，系统评估标签噪声校正技术在确保基于有偏数据集训练的模型公平性方面的有效性。该方法论涉及对标签噪声量的操控，既能用于公平性基准测试，也可应用于标准机器学习数据集。我们应用该方法论，根据多项公平性指标，在标准OpenML数据集上分析了六种标签噪声校正方法。结果表明，混合标签噪声校正方法在预测性能与公平性之间取得了最佳平衡。基于聚类的校正方法能够最大程度地减少歧视，但代价是预测性能较低。