Machine learning models are often brittle under distribution shift, i.e., when data distributions at test time differ from those during training. Understanding this failure mode is central to identifying and mitigating safety risks of mass adoption of machine learning. Here we analyze ridge regression under concept shift -- a form of distribution shift in which the input-label relationship changes at test time. We derive an exact expression for prediction risk in the high-dimensional limit. Our results reveal nontrivial effects of concept shift on generalization performance, depending on the properties of robust and nonrobust features of the input. We show that test performance can exhibit a nonmonotonic data dependence, even when double descent is absent. Finally, our experiments on MNIST and FashionMNIST suggest that this intriguing behavior is present also in classification problems.

翻译：机器学习模型在分布漂移下通常表现脆弱，即测试时的数据分布与训练时不同。理解这种失效模式对于识别和缓解大规模采用机器学习的安全风险至关重要。本文分析了概念漂移下的岭回归——这是一种测试时输入-标签关系发生变化的分布漂移形式。我们推导了高维极限下预测风险的精确表达式。结果表明，概念漂移对泛化性能的影响取决于输入特征的鲁棒性与非鲁棒性属性，呈现出非平凡效应。我们证明即使不存在双下降现象，测试性能也可能表现出非单调的数据依赖性。最后，我们在MNIST和FashionMNIST数据集上的实验表明，这种引人注目的行为在分类问题中同样存在。