Machine learning models are vulnerable to adversarial perturbations, and a thought-provoking paper by Bubeck and Sellke has analyzed this phenomenon through the lens of over-parameterization: interpolating smoothly the data requires significantly more parameters than simply memorizing it. However, this "universal" law provides only a necessary condition for robustness, and it is unable to discriminate between models. In this paper, we address these gaps by focusing on empirical risk minimization in two prototypical settings, namely, random features and the neural tangent kernel (NTK). We prove that, for random features, the model is not robust for any degree of over-parameterization, even when the necessary condition coming from the universal law of robustness is satisfied. In contrast, for even activations, the NTK model meets the universal lower bound, and it is robust as soon as the necessary condition on over-parameterization is fulfilled. This also addresses a conjecture in prior work by Bubeck, Li and Nagaraj. Our analysis decouples the effect of the kernel of the model from an "interaction matrix", which describes the interaction with the test data and captures the effect of the activation. Our theoretical results are corroborated by numerical evidence on both synthetic and standard datasets (MNIST, CIFAR-10).

翻译：机器学习模型易受对抗性扰动影响，Bubeck与Sellke一篇引人深思的论文通过过参数化视角分析了这一现象：平滑插值数据所需的参数数量远多于单纯记忆数据。然而，这一"普适"定律仅提供了鲁棒性的必要条件，无法区分不同模型。本文通过聚焦两个典型场景——随机特征与神经切线核（NTK）中的经验风险最小化，填补了上述空白。我们证明：对于随机特征，无论过参数化程度如何，模型均不具备鲁棒性，即使满足普适鲁棒定律推导的必要条件；相反，对于偶激活函数，NTK模型满足普适下界，且一旦满足过参数化的必要条件即具有鲁棒性。这一结果亦回应了Bubeck、Li与Nagaraj先前研究中的猜想。我们的分析将模型核效应与"交互矩阵"解耦——该矩阵描述模型与测试数据的相互作用，并捕获激活函数的影响。理论结果通过合成数据集与标准数据集（MNIST、CIFAR-10）上的数值实验得到验证。