Covariate distribution shifts and adversarial perturbations present robustness challenges to the conventional statistical learning framework: mild shifts in the test covariate distribution can significantly affect the performance of the statistical model learned based on the training distribution. The model performance typically deteriorates when extrapolation happens: namely, covariates shift to a region where the training distribution is scarce, and naturally, the learned model has little information. For robustness and regularization considerations, adversarial perturbation techniques are proposed as a remedy; however, careful study needs to be carried out about what extrapolation region adversarial covariate shift will focus on, given a learned model. This paper precisely characterizes the extrapolation region, examining both regression and classification in an infinite-dimensional setting. We study the implications of adversarial covariate shifts to subsequent learning of the equilibrium -- the Bayes optimal model -- in a sequential game framework. We exploit the dynamics of the adversarial learning game and reveal the curious effects of the covariate shift to equilibrium learning and experimental design. In particular, we establish two directional convergence results that exhibit distinctive phenomena: (1) a blessing in regression, the adversarial covariate shifts in an exponential rate to an optimal experimental design for rapid subsequent learning; (2) a curse in classification, the adversarial covariate shifts in a subquadratic rate to the hardest experimental design trapping subsequent learning.

翻译：协变量分布偏移和对抗扰动对传统统计学习框架提出了鲁棒性挑战：测试协变量分布的轻微偏移可能会显著影响基于训练分布学习的统计模型的性能。当外推发生时，模型性能通常会下降：即协变量偏移到训练分布稀疏的区域，自然导致所学模型信息匮乏。出于鲁棒性和正则化的考虑，对抗扰动技术被提出作为补救措施；然而，需要仔细研究的是，给定一个学到的模型，对抗性协变量偏移将聚焦于何种外推区域。本文在无限维设置下精确刻画了这一外推区域，同时考察了回归和分类问题。我们在序列博弈框架下研究了对抗性协变量偏移对后续学习均衡——贝叶斯最优模型——的影响。我们利用对抗学习博弈的动态特性，揭示了协变量偏移对均衡学习和实验设计的奇特影响。特别是，我们建立了两个方向收敛结果，展现了截然不同的现象：(1) 回归中的“福音”：对抗性协变量偏移以指数速率收敛至最优实验设计，从而促进后续快速学习；(2) 分类中的“诅咒”：对抗性协变量偏移以次二次速率收敛至最困难的实验设计，从而阻碍后续学习。