A pervasive phenomenon in machine learning applications is distribution shift, where training and deployment conditions for a machine learning model differ. As distribution shift typically results in a degradation in performance, much attention has been devoted to algorithmic interventions that mitigate these detrimental effects. In this paper, we study the effect of distribution shift in the presence of model misspecification, specifically focusing on $L_{\infty}$-misspecified regression and adversarial covariate shift, where the regression target remains fixed while the covariate distribution changes arbitrarily. We show that empirical risk minimization, or standard least squares regression, can result in undesirable misspecification amplification where the error due to misspecification is amplified by the density ratio between the training and testing distributions. As our main result, we develop a new algorithm -- inspired by robust optimization techniques -- that avoids this undesirable behavior, resulting in no misspecification amplification while still obtaining optimal statistical rates. As applications, we use this regression procedure to obtain new guarantees in offline and online reinforcement learning with misspecification and establish new separations between previously studied structural conditions and notions of coverage.

翻译：机器学习应用中一个普遍存在的现象是分布偏移，即机器学习模型的训练条件与部署条件存在差异。由于分布偏移通常会导致性能下降，因此大量研究致力于开发能够减轻这些不利影响的算法干预措施。在本文中，我们研究了模型错误设定情况下分布偏移的影响，特别关注$L_{\infty}$-错误设定回归和对抗性协变量偏移，其中回归目标保持不变而协变量分布任意变化。我们证明，经验风险最小化或标准最小二乘回归可能导致不良的错误设定放大现象，即由错误设定引起的误差被训练分布与测试分布之间的密度比放大。作为主要结果，我们提出了一种受鲁棒优化技术启发的新算法——该算法避免了这种不良行为，在实现无错误设定放大的同时仍能获得最优统计速率。在应用方面，我们利用该回归过程为存在错误设定的离线与在线强化学习提供了新的理论保证，并建立了先前研究的结构性条件与覆盖概念之间的新区分。