Current AI regulations require discarding sensitive features (e.g., gender, race, religion) in the algorithm's decision-making process to prevent unfair outcomes. However, even without sensitive features in the training set, algorithms can persist in discrimination. Indeed, when sensitive features are omitted (fairness under unawareness), they could be inferred through non-linear relations with the so called proxy features. In this work, we propose a way to reveal the potential hidden bias of a machine learning model that can persist even when sensitive features are discarded. This study shows that it is possible to unveil whether the black-box predictor is still biased by exploiting counterfactual reasoning. In detail, when the predictor provides a negative classification outcome, our approach first builds counterfactual examples for a discriminated user category to obtain a positive outcome. Then, the same counterfactual samples feed an external classifier (that targets a sensitive feature) that reveals whether the modifications to the user characteristics needed for a positive outcome moved the individual to the non-discriminated group. When this occurs, it could be a warning sign for discriminatory behavior in the decision process. Furthermore, we leverage the deviation of counterfactuals from the original sample to determine which features are proxies of specific sensitive information. Our experiments show that, even if the model is trained without sensitive features, it often suffers discriminatory biases.

翻译：当前人工智能监管要求算法决策过程中剔除敏感特征（如性别、种族、宗教）以防止不公平结果。然而，即使训练集中不包含敏感特征，算法仍可能持续存在歧视行为。事实上，当敏感特征被省略时（公平无感知环境），这些特征可能通过与所谓代理特征的非线性关系被推断出来。本研究提出一种方法，用于揭示机器学习模型在剔除敏感特征后仍可能存在的潜在隐藏偏差。研究表明，通过运用反事实推理，能够揭示黑箱预测器是否仍存在偏差。具体而言，当预测器给出负面分类结果时，我们的方法首先为受歧视用户类别构建反事实样本以获得正面结果。随后，将相同的反事实样本输入针对敏感特征的外部分类器，以判定为了获得正面结果而对用户特征进行的修改是否将个体转移至非受歧视群体。当这种情况发生时，可能标志着决策过程中存在歧视行为的警示信号。此外，我们利用反事实样本与原始样本的偏差程度来确定哪些特征是特定敏感信息的代理变量。实验表明，即使模型训练时未使用敏感特征，其仍常遭受歧视性偏差的影响。