Covariate distribution shifts and adversarial perturbations present robustness challenges to the conventional statistical learning framework: mild shifts in the test covariate distribution can significantly affect the performance of the statistical model learned based on the training distribution. The model performance typically deteriorates when extrapolation happens: namely, covariates shift to a region where the training distribution is scarce, and naturally, the learned model has little information. For robustness and regularization considerations, adversarial perturbation techniques are proposed as a remedy; however, careful study needs to be carried out about what extrapolation region adversarial covariate shift will focus on, given a learned model. This paper precisely characterizes the extrapolation region, examining both regression and classification in an infinite-dimensional setting. We study the implications of adversarial covariate shifts to subsequent learning of the equilibrium -- the Bayes optimal model -- in a sequential game framework. We exploit the dynamics of the adversarial learning game and reveal the curious effects of the covariate shift to equilibrium learning and experimental design. In particular, we establish two directional convergence results that exhibit distinctive phenomena: (1) a blessing in regression, the adversarial covariate shifts in an exponential rate to an optimal experimental design for rapid subsequent learning, (2) a curse in classification, the adversarial covariate shifts in a subquadratic rate fast to the hardest experimental design trapping subsequent learning.

翻译：协变量分布偏移和对抗扰动对传统统计学习框架构成了鲁棒性挑战：测试协变量分布的轻微偏移可能显著影响基于训练分布学习的统计模型的性能。当外推发生时，模型性能通常会恶化：即协变量偏移到训练分布稀少的区域，自然导致学习模型缺乏信息。出于鲁棒性和正则化的考虑，对抗扰动技术被提出作为补救措施；然而，需要仔细研究在给定学习模型的情况下，对抗协变量偏移将集中于哪个外推区域。本文精确刻画了该外推区域，在无限维设置下检验了回归和分类问题。我们在序贯博弈框架中研究了对抗协变量偏移对后续学习均衡——贝叶斯最优模型——的影响。我们利用对抗学习博弈的动力学，揭示了协变量偏移对均衡学习和实验设计的奇特效应。特别地，我们确立了两种表现出独特现象的方向性收敛结果：(1) 回归中的恩赐：对抗协变量偏移以指数速率收敛到最优实验设计，促进了后续快速学习；(2) 分类中的诅咒：对抗协变量偏移以次二次速率快速收敛到最困难的实验设计，阻碍了后续学习。