We study the robust interpolation problem of arbitrary data distributions supported on a bounded space and propose a two-fold law of robustness. Robust interpolation refers to the problem of interpolating $n$ noisy training data points in $\mathbb{R}^d$ by a Lipschitz function. Although this problem has been well understood when the samples are drawn from an isoperimetry distribution, much remains unknown concerning its performance under generic or even the worst-case distributions. We prove a Lipschitzness lower bound $\Omega(\sqrt{n/p})$ of the interpolating neural network with $p$ parameters on arbitrary data distributions. With this result, we validate the law of robustness conjecture in prior work by Bubeck, Li, and Nagaraj on two-layer neural networks with polynomial weights. We then extend our result to arbitrary interpolating approximators and prove a Lipschitzness lower bound $\Omega(n^{1/d})$ for robust interpolation. Our results demonstrate a two-fold law of robustness: i) we show the potential benefit of overparametrization for smooth data interpolation when $n=\mathrm{poly}(d)$, and ii) we disprove the potential existence of an $O(1)$-Lipschitz robust interpolating function when $n=\exp(\omega(d))$.

翻译：我们研究了有界空间上任意数据分布的鲁棒插值问题，并提出了一个双重鲁棒性定律。鲁棒插值是指通过Lipschitz函数对$\mathbb{R}^d$中的$n$个含噪训练数据点进行插值的问题。尽管当样本来自等周分布时该问题已得到充分理解，但在一般分布甚至最坏情况分布下的性能仍有许多未知之处。我们证明，对于任意数据分布，具有$p$个参数的插值神经网络的Lipschitz下界为$\Omega(\sqrt{n/p})$。基于这一结果，我们验证了Bubeck、Li和Nagaraj先前工作中关于具有多项式权重的两层神经网络的鲁棒性定律猜想。随后，我们将结果推广到任意插值近似器，并证明了鲁棒插值的Lipschitz下界$\Omega(n^{1/d})$。我们的结果展示了双重鲁棒性定律：i) 当$n=\mathrm{poly}(d)$时，我们展示了过参数化对平滑数据插值的潜在优势；ii) 当$n=\exp(\omega(d))$时，我们否定了存在$O(1)$-Lipschitz鲁棒插值函数的可能性。