To mitigate unfair and unethical discrimination over sensitive features (e.g., gender, age, or race), fairness testing plays an integral role in engineering systems that leverage AI models to handle tabular data. A key challenge therein is how to effectively reveal fairness bugs under an intractable sample size using perturbation. Much current work has been focusing on designing the test sample generators, ignoring the valuable knowledge about data characteristics that can help guide the perturbation and hence limiting their full potential. In this paper, we seek to bridge such a gap by proposing a generic framework of causally perturbed fairness testing, dubbed CausalFT. Through causal inference, the key idea of CausalFT is to extract the most directly and causally relevant non-sensitive feature to its sensitive counterpart, which can jointly influence the prediction of the label. Such a causal relationship is then seamlessly injected into the perturbation to guide a test sample generator. Unlike existing generator-level work, CausalFT serves as a higher-level framework that can be paired with diverse base generators. Extensive experiments on 1296 cases confirm that CausalFT can considerably improve arbitrary base generators in revealing fairness bugs over 93% of the cases with acceptable extra runtime overhead. Compared with a state-of-the-art approach that ranks the non-sensitive features solely based on correlation, CausalFT performs significantly better on 64% cases while being much more efficient. Further, CausalFT can better improve bias resilience in nearly all cases.

翻译：为减轻基于敏感特征（如性别、年龄或种族）的不公平及不道德的歧视，公平性测试在利用人工智能模型处理表格数据的工程系统中扮演着不可或缺的角色。其中的一个关键挑战在于如何通过扰动在样本量难以处理的情况下有效揭示公平性缺陷。当前许多工作主要聚焦于设计测试样本生成器，却忽略了有助于指导扰动的数据特征宝贵知识，从而限制了其全部潜力。本文旨在通过提出一个通用的因果扰动公平性测试框架（命名为CausalFT）来弥合这一差距。CausalFT的核心思想是通过因果推断，提取与敏感特征最直接且因果相关的非敏感特征，二者可共同影响标签的预测。随后，这种因果关系被无缝注入扰动中，以指导测试样本生成器。与现有的生成器层面工作不同，CausalFT作为一个更高层次的框架，可与多种基础生成器配对使用。在1296个案例上的广泛实验证实，CausalFT能显著提升任意基础生成器在揭示公平性缺陷方面的性能，在超过93%的案例中有效，且额外运行时开销可接受。与仅基于相关性对非敏感特征进行排序的先进方法相比，CausalFT在64%的案例中表现显著更优，同时效率更高。此外，CausalFT在几乎所有案例中都能更好地提升偏差鲁棒性。