We study the error of linear regression in the face of adversarial attacks. In this framework, an adversary changes the input to the regression model in order to maximize the prediction error. We provide bounds on the prediction error in the presence of an adversary as a function of the parameter norm and the error in the absence of such an adversary. We show how these bounds make it possible to study the adversarial error using analysis from non-adversarial setups. The obtained results shed light on the robustness of overparameterized linear models to adversarial attacks. Adding features might be either a source of additional robustness or brittleness. On the one hand, we use asymptotic results to illustrate how double-descent curves can be obtained for the adversarial error. On the other hand, we derive conditions under which the adversarial error can grow to infinity as more features are added, while at the same time, the test error goes to zero. We show this behavior is caused by the fact that the norm of the parameter vector grows with the number of features. It is also established that $\ell_\infty$ and $\ell_2$-adversarial attacks might behave fundamentally differently due to how the $\ell_1$ and $\ell_2$-norms of random projections concentrate. We also show how our reformulation allows for solving adversarial training as a convex optimization problem. This fact is then exploited to establish similarities between adversarial training and parameter-shrinking methods and to study how the training might affect the robustness of the estimated models.

翻译：我们在对抗攻击的框架下研究线性回归的误差。在该框架中，攻击者通过改变回归模型的输入以最大化预测误差。我们给出了存在攻击者时预测误差的上下界，将其表示为参数范数以及无攻击者时误差的函数。我们展示了这些界如何使得利用非对抗场景的分析方法研究对抗误差成为可能。所得结果揭示了过参数化线性模型对对抗攻击的鲁棒性。增加特征既可能带来额外的鲁棒性，也可能导致脆弱性。一方面，我们利用渐近结果说明了如何获得对抗误差的双下降曲线。另一方面，我们推导出在某些条件下，当特征数量增加时对抗误差可能趋于无穷大，而与此同时测试误差却趋于零。我们证明这一行为是由参数向量范数随特征数量增长这一事实所导致。另外，由于随机投影的$\ell_1$范数与$\ell_2$范数的集中行为不同，$\ell_\infty$和$\ell_2$-对抗攻击可能在根本上有不同的表现。我们还展示了我们的重新表述如何允许将对抗训练转化为一个凸优化问题。随后利用这一事实建立对抗训练与参数收缩方法之间的相似性，并研究训练过程可能如何影响估计模型的鲁棒性。