We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-distribution prediction, where the test distribution deviates arbitrarily from the train distribution. We establish general conditions that determine the sign of the optimal regularization level under covariate and regression shifts. These conditions capture the alignment between the covariance and signal structures in the train and test data and reveal stark differences compared to the in-distribution setting. For example, a negative regularization level can be optimal under covariate shift or regression shift, even when the training features are isotropic or the design is underparameterized. Furthermore, we prove that the optimally-tuned risk is monotonic in the data aspect ratio, even in the out-of-distribution setting and when optimizing over negative regularization levels. In general, our results do not make any modeling assumptions for the train or the test distributions, except for moment bounds, and allow for arbitrary shifts and the widest possible range of (negative) regularization levels.

翻译：我们研究了分布外预测中（测试分布与训练分布任意偏离）最优岭回归正则化及最优岭回归风险的行为。我们建立了决定协变量偏移和回归偏移下最优正则化水平符号的通用条件。这些条件捕捉了训练与测试数据中协方差与信号结构之间的对齐关系，并揭示了与分布内设置相比的显著差异。例如，在协变量偏移或回归偏移下，即使训练特征具有各向同性或设计欠参数化，负正则化水平也可能达到最优。此外，我们证明即使在分布外设置中以及优化负正则化水平时，最优调整后的风险在数据纵横比上仍是单调的。总体上，我们的结果除矩有界性外，未对训练或测试分布做出任何建模假设，并允许任意偏移和最广泛（负）正则化水平范围。