Practitioners often deploy a learned prediction model in a new environment where the joint distribution of covariate and response has shifted. In observational data, the distribution shift is often driven by unobserved confounding factors lurking in the environment, with the underlying mechanism unknown. Confounding can obfuscate the definition of the best prediction model (concept shift) and shift covariates to domains yet unseen (covariate shift). Therefore, a model maximizing prediction accuracy in the source environment could suffer a significant accuracy drop in the target environment. This motivates us to study the domain adaptation problem with observational data: given labeled covariate and response pairs from a source environment, and unlabeled covariates from a target environment, how can one predict the missing target response reliably? We root the adaptation problem in a linear structural causal model to address endogeneity and unobserved confounding. We study the necessity and benefit of leveraging exogenous, invariant covariate representations to cure concept shifts and improve target prediction. This further motivates a new representation learning method for adaptation that optimizes for a lower-dimensional linear subspace and, subsequently, a prediction model confined to that subspace. The procedure operates on a non-convex objective-that naturally interpolates between predictability and stability/invariance-constrained on the Stiefel manifold. We study the optimization landscape and prove that, when the regularization is sufficient, nearly all local optima align with an invariant linear subspace resilient to both concept and covariate shift. In terms of predictability, we show a model that uses the learned lower-dimensional subspace can incur a nearly ideal gap between target and source risk. Three real-world data sets are investigated to validate our method and theory.

翻译：实践中，学习者常将训练好的预测模型部署于协变量与响应变量的联合分布已发生漂移的新环境中。在观测数据中，分布漂移通常由环境中潜藏的未观测混杂因子驱动，其内在机制未知。混杂效应可能模糊最佳预测模型的定义（概念漂移），并将协变量迁移至尚未观测的域（协变量漂移）。因此，在源环境中预测精度最优的模型，在目标环境中可能遭遇显著的精度下降。这促使我们研究观测数据下的域适应问题：给定源环境中带标签的协变量-响应对，以及目标环境中未标记的协变量，如何可靠地预测缺失的目标响应？我们将适应问题植根于线性结构因果模型，以处理内生性与未观测混杂。我们研究了利用外生、不变的协变量表示来修正概念漂移并提升目标预测的必要性与优势。这进一步启发了一种新的适应表征学习方法，该方法通过优化得到一个低维线性子空间，并随后将预测模型约束于该子空间内。该过程基于一个非凸目标函数进行优化——该函数自然地在可预测性与稳定性/不变性之间进行权衡——并受限于Stiefel流形约束。我们分析了优化景观并证明：当正则化充分时，几乎所有局部最优解都对应于一个对概念漂移与协变量漂移均具有鲁棒性的不变线性子空间。在可预测性方面，我们证明了使用习得的低维子空间的模型，其目标风险与源风险之间的差距可接近理想界限。我们通过三个真实世界数据集验证了所提方法与理论。