While deep learning has become a core functional module of most software systems, concerns regarding the fairness of ML predictions have emerged as a significant issue that affects prediction results due to discrimination. Intersectional bias, which disproportionately affects members of subgroups, is a prime example of this. For instance, a machine learning model might exhibit bias against darker-skinned women, while not showing bias against individuals with darker skin or women. This problem calls for effective fairness testing before the deployment of such deep learning models in real-world scenarios. However, research into detecting such bias is currently limited compared to research on individual and group fairness. Existing tools to investigate intersectional bias lack important features such as support for multiple fairness metrics, fast and efficient computation, and user-friendly interpretation. This paper introduces Fairpriori, a novel biased subgroup discovery method, which aims to address these limitations. Fairpriori incorporates the frequent itemset generation algorithm to facilitate effective and efficient investigation of intersectional bias by producing fast fairness metric calculations on subgroups of a dataset. Through comparison with the state-of-the-art methods (e.g., Themis, FairFictPlay, and TestSGD) under similar conditions, Fairpriori demonstrates superior effectiveness and efficiency when identifying intersectional bias. Specifically, Fairpriori is easier to use and interpret, supports a wider range of use cases by accommodating multiple fairness metrics, and exhibits higher efficiency in computing fairness metrics. These findings showcase Fairpriori's potential for effectively uncovering subgroups affected by intersectional bias, supported by its open-source tooling at https://anonymous.4open.science/r/Fairpriori-0320.
翻译:尽管深度学习已成为大多数软件系统的核心功能模块,但机器学习预测的公平性问题已成为影响预测结果的重要议题,其中歧视是主要原因。交叉偏见(intersectional bias)不成比例地影响子群成员,是这一问题的典型例证。例如,机器学习模型可能对深肤色女性表现出偏见,而对深肤色个体或女性整体却未显示偏见。这要求在将此类深度学习模型部署到实际场景前进行有效的公平性测试。然而,与个体公平性和群体公平性的研究相比,目前检测此类偏见的研究仍较为有限。现有探究交叉偏见的工具缺乏重要特性,包括多公平性指标支持、快速高效的计算能力以及用户友好的结果解读。本文提出Fairpriori——一种新颖的有偏子群发现方法,旨在解决这些局限性。Fairpriori整合了频繁项集生成算法,通过对数据集的子群进行快速公平性指标计算,促进对交叉偏见的高效研究。在与现有先进方法(如Themis、FairFictPlay和TestSGD)在相似条件下的比较中,Fairpriori在识别交叉偏见方面展现出更优的效能与效率。具体而言,Fairpriori更易于使用和解释,通过适配多种公平性指标支持更广泛的应用场景,并在计算公平性指标时表现出更高的效率。这些发现展示了Fairpriori在有效揭示受交叉偏见影响子群方面的潜力,其开源工具支持可见于https://anonymous.4open.science/r/Fairpriori-0320。