Machine learning systems are often applied to data that is drawn from a different distribution than the training distribution. Recent work has shown that for a variety of classification and signal reconstruction problems, the out-of-distribution performance is strongly linearly correlated with the in-distribution performance. If this relationship or more generally a monotonic one holds, it has important consequences. For example, it allows to optimize performance on one distribution as a proxy for performance on the other. In this paper, we study conditions under which a monotonic relationship between the performances of a model on two distributions is expected. We prove an exact asymptotic linear relation for squared error and a monotonic relation for misclassification error for ridge-regularized general linear models under covariate shift, as well as an approximate linear relation for linear inverse problems.

翻译：机器学习系统通常应用于与训练分布不同的数据分布。近期研究表明，在多种分类与信号重构问题中，分布外性能与分布内性能存在强线性相关。若这种关系（或更普遍的单调关系）成立，将产生重要影响。例如，它允许以某一分布上的性能优化作为另一分布性能的代理指标。本文研究了模型在两个分布上的性能之间预期存在单调关系的条件。我们证明了：在协变量漂移下，岭正则化广义线性模型的平方误差具有精确渐近线性关系，误分类误差存在单调关系；同时，线性逆问题存在近似线性关系。