Fair face recognition is all about learning invariant feature that generalizes to unseen faces in any demographic group. Unfortunately, face datasets inevitably capture the imbalanced demographic attributes that are ubiquitous in real-world observations, and the model learns biased feature that generalizes poorly in the minority group. We point out that the bias arises due to the confounding demographic attributes, which mislead the model to capture the spurious demographic-specific feature. The confounding effect can only be removed by causal intervention, which requires the confounder annotations. However, such annotations can be prohibitively expensive due to the diversity of the demographic attributes. To tackle this, we propose to generate diverse data partitions iteratively in an unsupervised fashion. Each data partition acts as a self-annotated confounder, enabling our Invariant Feature Regularization (INV-REG) to deconfound. INV-REG is orthogonal to existing methods, and combining INV-REG with two strong baselines (Arcface and CIFP) leads to new state-of-the-art that improves face recognition on a variety of demographic groups. Code is available at https://github.com/PanasonicConnect/InvReg.

翻译：公平人脸识别旨在学习能够泛化到任意人口群体中未见人脸的不变特征。然而，人脸数据集不可避免地捕捉到真实世界观察中普遍存在的人口属性不平衡现象，导致模型学习到对少数群体泛化性能较差的偏置特征。我们指出，这种偏置源于混杂的人口属性，这些属性误导模型捕获了虚假的群体特异性特征。混杂效应只能通过因果干预来消除，而这需要混杂因素的标注。但由于人口属性的多样性，此类标注成本可能极其高昂。为应对这一挑战，我们提出以无监督方式迭代生成多样化的数据分区。每个数据分区充当自标注的混杂因素，使我们的不变特征正则化（INV-REG）能够实现反混杂。INV-REG与现有方法正交，将其与两个强基线方法（Arcface和CIFP）结合后，在多种人口群体上实现了人脸识别性能的最新水平。代码开源地址：https://github.com/PanasonicConnect/InvReg。