Covariate shift in the test data is a common practical phenomena that can significantly downgrade both the accuracy and the fairness performance of the model. Ensuring fairness across different sensitive groups under covariate shift is of paramount importance due to societal implications like criminal justice. We operate in the unsupervised regime where only a small set of unlabeled test samples along with a labeled training set is available. Towards improving fairness under this highly challenging yet realistic scenario, we make three contributions. First is a novel composite weighted entropy based objective for prediction accuracy which is optimized along with a representation matching loss for fairness. We experimentally verify that optimizing with our loss formulation outperforms a number of state-of-the-art baselines in the pareto sense with respect to the fairness-accuracy tradeoff on several standard datasets. Our second contribution is a new setting we term Asymmetric Covariate Shift that, to the best of our knowledge, has not been studied before. Asymmetric covariate shift occurs when distribution of covariates of one group shifts significantly compared to the other groups and this happens when a dominant group is over-represented. While this setting is extremely challenging for current baselines, We show that our proposed method significantly outperforms them. Our third contribution is theoretical, where we show that our weighted entropy term along with prediction loss on the training set approximates test loss under covariate shift. Empirically and through formal sample complexity bounds, we show that this approximation to the unseen test loss does not depend on importance sampling variance which affects many other baselines.

翻译：测试数据中的协变量偏移是一种常见的实际现象，可能显著降低模型的准确性及公平性表现。由于涉及刑事司法等社会影响，确保协变量偏移下不同敏感群体间的公平性至关重要。我们研究的是无监督场景，仅有少量无标签测试样本和带标签的训练集可用。为了在这一极具挑战性但现实的情景中提升公平性，我们做出了三项贡献。首先，我们提出了一种基于复合加权熵的新型预测准确性目标函数，并与用于公平性的表示匹配损失联合优化。实验验证表明，在多个标准数据集上，使用我们的损失函数进行优化在帕累托意义上超越了若干最先进基线方法的公平性-准确性权衡表现。其次，我们提出了一个名为“非对称协变量偏移”的新设定——据我们所知，该设定此前未被研究过。当某一群体的协变量分布相较其他群体发生显著偏移时，即主导群体过度代表的情况下，就会出现非对称协变量偏移。尽管这一设定对现有基线方法极具挑战性，但我们证明所提出的方法显著优于它们。第三项贡献是理论层面的：我们证明，在协变量偏移下，训练集上的加权熵项与预测损失之和可近似测试损失。通过实验及形式化的样本复杂度界限，我们表明这种对未知测试损失的近似不依赖于影响许多其他基线方法的重要性采样方差。