In this paper, we study the generalization performance of min $\ell_2$-norm overfitting solutions for the neural tangent kernel (NTK) model of a two-layer neural network with ReLU activation that has no bias term. We show that, depending on the ground-truth function, the test error of overfitted NTK models exhibits characteristics that are different from the "double-descent" of other overparameterized linear models with simple Fourier or Gaussian features. Specifically, for a class of learnable functions, we provide a new upper bound of the generalization error that approaches a small limiting value, even when the number of neurons $p$ approaches infinity. This limiting value further decreases with the number of training samples $n$. For functions outside of this class, we provide a lower bound on the generalization error that does not diminish to zero even when $n$ and $p$ are both large.

翻译：本文研究了具有ReLU激活函数且无偏置项的双层神经网络神经正切核模型的极小$\ell_2$范数过拟合解的泛化性能。研究表明，取决于真实函数，过拟合NTK模型的测试误差特征与使用简单傅里叶或高斯特征的其他过参数化线性模型的“双下降”现象有所不同。具体而言，对于一类可学习函数，我们给出了泛化误差的新上界，该上界在神经元数量$p$趋于无穷时趋近于一个小的极限值，且该极限值随训练样本数量$n$的增加而进一步减小。对于该类之外的函数，我们给出了泛化误差的下界，该下界即使在$n$和$p$均较大时也不会衰减至零。