The increasing use of Machine Learning (ML) software can lead to unfair and unethical decisions, thus fairness bugs in software are becoming a growing concern. Addressing these fairness bugs often involves sacrificing ML performance, such as accuracy. To address this issue, we present a novel counterfactual approach that uses counterfactual thinking to tackle the root causes of bias in ML software. In addition, our approach combines models optimized for both performance and fairness, resulting in an optimal solution in both aspects. We conducted a thorough evaluation of our approach on 10 benchmark tasks using a combination of 5 performance metrics, 3 fairness metrics, and 15 measurement scenarios, all applied to 8 real-world datasets. The conducted extensive evaluations show that the proposed method significantly improves the fairness of ML software while maintaining competitive performance, outperforming state-of-the-art solutions in 84.6% of overall cases based on a recent benchmarking tool.
翻译:随着机器学习(ML)软件的广泛应用,不公平和不道德的决策问题日益凸显,软件中的公平性缺陷已成为亟待关注的议题。解决这些公平性缺陷通常需要牺牲ML性能(如准确率)。针对此问题,我们提出了一种创新的反事实方法,通过反事实思维从根本上消除ML软件中的偏差根源。此外,本方法融合了兼顾性能与公平性的优化模型,从而在两方面均实现最优解。我们在10个基准任务上进行了全面评估,采用5项性能指标、3项公平性指标及15种测量场景的组合,所有测试均基于8个真实数据集完成。广泛评估表明:所提方法在显著提升ML软件公平性的同时保持了富有竞争力的性能,基于最新基准工具的比较中,该方法在84.6%的整体案例中优于现有最优解决方案。