Domain adaptation (DA) is a statistical learning problem that arises when the distribution of the source data used to train a model differs from that of the target data used to evaluate the model. While many DA algorithms have demonstrated considerable empirical success, blindly applying these algorithms can often lead to worse performance on new datasets. To address this, it is crucial to clarify the assumptions under which a DA algorithm has good target performance. In this work, we focus on the assumption of the presence of conditionally invariant components (CICs), which are relevant for prediction and remain conditionally invariant across the source and target data. We demonstrate that CICs, which can be estimated through conditional invariant penalty (CIP), play three prominent roles in providing target risk guarantees in DA. First, we propose a new algorithm based on CICs, importance-weighted conditional invariant penalty (IW-CIP), which has target risk guarantees beyond simple settings such as covariate shift and label shift. Second, we show that CICs help identify large discrepancies between source and target risks of other DA algorithms. Finally, we demonstrate that incorporating CICs into the domain invariant projection (DIP) algorithm can address its failure scenario caused by label-flipping features. We support our new algorithms and theoretical findings via numerical experiments on synthetic data, MNIST, CelebA, Camelyon17, and DomainNet datasets.

翻译：领域自适应（DA）是一个统计学习问题，当用于训练模型的源数据分布与用于评估模型的目标数据分布不同时出现。尽管许多DA算法已展现出显著的实证成功，但盲目应用这些算法往往会导致在新数据集上的性能下降。为解决这一问题，阐明DA算法在何种假设下具有良好目标性能至关重要。在本研究中，我们重点关注条件不变分量（CICs）存在的假设，这些分量与预测相关且在源数据和目标数据间保持条件不变。我们证明，通过条件不变惩罚（CIP）可估计的CICs在提供DA目标风险保证方面发挥三个突出作用。首先，我们提出一种基于CICs的新算法——重要性加权条件不变惩罚（IW-CIP），该算法在协变量偏移和标签偏移等简单设定之外仍具有目标风险保证。其次，我们表明CICs有助于识别其他DA算法在源风险与目标风险间的巨大差异。最后，我们证明将CICs融入领域不变投影（DIP）算法可以解决由标签翻转特征引起的失效场景。我们通过在合成数据、MNIST、CelebA、Camelyon17和DomainNet数据集上的数值实验，验证了新算法和理论发现。