Domain adaptation (DA) is a statistical learning problem that arises when the distribution of the source data used to train a model differs from that of the target data used to evaluate the model. While many DA algorithms have demonstrated considerable empirical success, blindly applying these algorithms can often lead to worse performance on new datasets. To address this, it is crucial to clarify the assumptions under which a DA algorithm has good target performance. In this work, we focus on the assumption of the presence of conditionally invariant components (CICs), which are relevant for prediction and remain conditionally invariant across the source and target data. We demonstrate that CICs, which can be estimated through conditional invariant penalty (CIP), play three prominent roles in providing target risk guarantees in DA. First, we propose a new algorithm based on CICs, importance-weighted conditional invariant penalty (IW-CIP), which has target risk guarantees beyond simple settings such as covariate shift and label shift. Second, we show that CICs help identify large discrepancies between source and target risks of other DA algorithms. Finally, we demonstrate that incorporating CICs into the domain invariant projection (DIP) algorithm can address its failure scenario caused by label-flipping features. We support our new algorithms and theoretical findings via numerical experiments on synthetic data, MNIST, CelebA, and Camelyon17 datasets.

翻译：域自适应（DA）是一个统计学习问题，当用于训练模型的源数据分布与用于评估模型的目标数据分布不同时出现。尽管许多DA算法已展现出显著的经验成功，但盲目应用这些算法往往会导致在新数据集上的性能下降。为解决这一问题，明确DA算法具有良好目标性能的假设条件至关重要。本文聚焦于条件不变组件（CICs）存在的假设，这些组件与预测相关且在源域和目标域间保持条件不变。我们证明，可通过条件不变惩罚（CIP）估计的CICs在DA的目标风险保障中发挥三大突出作用。首先，我们提出一种基于CICs的新算法——重要性加权条件不变惩罚（IW-CIP），其在协变量偏移和标签偏移等简单设置之外仍具有目标风险保障。其次，我们证明CICs有助于识别其他DA算法中源域与目标域风险之间的显著差异。最后，我们论证将CICs纳入域不变投影（DIP）算法可解决其因标签翻转特征导致的失效场景。通过合成数据、MNIST、CelebA和Camelyon17数据集上的数值实验，我们为新算法及理论发现提供了支持。