Many existing approaches to generalizing statistical inference amidst distribution shift operate under the covariate shift assumption, which posits that the conditional distribution of unobserved variables given observable ones is invariant across populations. However, recent empirical investigations have demonstrated that adjusting for shift in observed variables (covariate shift) is often insufficient for generalization. In other words, covariate shift does not typically ``explain away'' the distribution shift between settings. As such, addressing the unknown yet non-negligible shift in the unobserved variables given observed ones (conditional shift) is crucial for generalizable inference. In this paper, we present a series of empirical evidence from two large-scale multi-site replication studies to support a new role of covariate shift in ``predicting'' the strength of the unknown conditional shift. Analyzing 680 studies across 65 sites, we find that even though the conditional shift is non-negligible, its strength can often be bounded by that of the observable covariate shift. However, this pattern only emerges when the two sources of shifts are quantified by our proposed standardized, ``pivotal'' measures. We then interpret this phenomenon by connecting it to similar patterns that can be theoretically derived from a random distribution shift model. Finally, we demonstrate that exploiting the predictive role of covariate shift leads to reliable and efficient uncertainty quantification for target estimates in generalization tasks with partially observed data. Overall, our empirical and theoretical analyses suggest a new way to approach the problem of distributional shift, generalizability, and external validity.

翻译：许多现有的分布偏移下统计推断泛化方法在协变量偏移假设下运作，该假设认为给定可观测变量条件下未观测变量的条件分布在各总体间保持不变。然而，最近的实证研究表明，仅针对观测变量偏移（协变量偏移）进行调整通常不足以实现泛化。换言之，协变量偏移通常无法"解释"不同情境间的分布差异。因此，处理给定观测变量条件下未观测变量的未知但不可忽略的偏移（条件偏移）对于可泛化推断至关重要。本文通过两项大规模多中心重复性研究提供了一系列实证证据，以支持协变量偏移在"预测"未知条件偏移强度方面的新作用。通过分析65个研究中心的680项研究，我们发现尽管条件偏移不可忽略，但其强度往往可以通过可观测的协变量偏移强度进行界定。然而，这种模式仅当使用我们提出的标准化"枢轴"度量对两种偏移源进行量化时才会显现。随后，我们通过将其与随机分布偏移模型中可理论推导的相似模式相联系，对这一现象进行了解释。最后，我们证明了利用协变量偏移的预测作用，能够在部分观测数据的泛化任务中为目标估计提供可靠且高效的不确定性量化。总体而言，我们的实证与理论分析为处理分布偏移、可泛化性及外部有效性问题提供了新的思路。