Fairness is crucial when training a deep-learning discriminative model, especially in the facial domain. Models tend to correlate specific characteristics (such as age and skin color) with unrelated attributes (downstream tasks), resulting in biases which do not correspond to reality. It is common knowledge that these correlations are present in the data and are then transferred to the models during training. This paper proposes a method to mitigate these correlations to improve fairness. To do so, we learn interpretable and meaningful paths lying in the semantic space of a pre-trained diffusion model (DiffAE) -- such paths being supervised by contrastive text dipoles. That is, we learn to edit protected characteristics (age and skin color). These paths are then applied to augment images to improve the fairness of a given dataset. We test the proposed method on CelebA-HQ and UTKFace on several downstream tasks with age and skin color as protected characteristics. As a proxy for fairness, we compute the difference in accuracy with respect to the protected characteristics. Quantitative results show how the augmented images help the model improve the overall accuracy, the aforementioned metric, and the disparity of equal opportunity. Code is available at: https://github.com/Moreno98/Vision-Language-Bias-Control.

翻译：公平性在训练深度学习判别模型时至关重要，尤其是在人脸领域。模型倾向于将特定特征（如年龄和肤色）与无关属性（下游任务）相关联，导致与现实不符的偏差。众所周知，这些相关性存在于数据中，并在训练过程中转移到模型上。本文提出一种缓解这些相关性以提升公平性的方法。为此，我们学习位于预训练扩散模型（DiffAE）语义空间中的可解释且有意义的路径——这些路径由对比文本偶极子监督。即，我们学习编辑受保护特征（年龄和肤色）。随后将这些路径应用于增强图像，以提升给定数据集的公平性。我们在CelebA-HQ和UTKFace数据集上，以年龄和肤色作为受保护特征，测试了所提方法在若干下游任务中的表现。作为公平性的代理指标，我们计算了与受保护特征相关的准确率差异。定量结果表明，增强图像有助于模型提升整体准确率、前述指标以及机会均等差异。代码发布于：https://github.com/Moreno98/Vision-Language-Bias-Control。