Deep neural networks (DNNs) typically involve a large number of parameters and are trained to achieve zero or near-zero training error. Despite such interpolation, they often exhibit strong generalization performance on unseen data, a phenomenon that has motivated extensive theoretical investigations. Comforting results show that interpolation indeed may not affect the minimax rate of convergence under the squared error loss. In the mean time, DNNs are well known to be highly vulnerable to adversarial perturbations in future inputs. A natural question then arises: Can interpolation also escape from suboptimal performance under a future $X$-attack? In this paper, we investigate the adversarial robustness of interpolating estimators in a framework of nonparametric regression. A finding is that interpolating estimators must be suboptimal even under a subtle future $X$-attack, and achieving perfect fitting can substantially damage their robustness. An interesting phenomenon in the high interpolation regime, which we term the curse of simple size, is also revealed and discussed. Numerical experiments support our theoretical findings.

翻译：深度神经网络（DNNs）通常包含大量参数，并被训练以达到零或接近零的训练误差。尽管存在这种插值现象，它们往往在未见数据上表现出强大的泛化性能，这一现象已激发了广泛的理论研究。令人欣慰的结果表明，在平方误差损失下，插值确实可能不影响收敛的极小极大速率。与此同时，众所周知，DNNs对未来输入中的对抗性扰动高度脆弱。一个自然的问题随之产生：插值是否也能在未来$X$-攻击下避免次优性能？在本文中，我们在非参数回归的框架下研究插值估计器的对抗鲁棒性。一个发现是，即使面对微妙的未来$X$-攻击，插值估计器也必然是次优的，实现完美拟合会严重损害其鲁棒性。我们还揭示并讨论了在高插值区域中出现的一个有趣现象，我们称之为“简单尺寸诅咒”。数值实验支持了我们的理论发现。