In this paper, we consider the generalization ability of deep wide feedforward ReLU neural networks defined on a bounded domain $\mathcal X \subset \mathbb R^{d}$. We first demonstrate that the generalization ability of the neural network can be fully characterized by that of the corresponding deep neural tangent kernel (NTK) regression. We then investigate on the spectral properties of the deep NTK and show that the deep NTK is positive definite on $\mathcal{X}$ and its eigenvalue decay rate is $(d+1)/d$. Thanks to the well established theories in kernel regression, we then conclude that multilayer wide neural networks trained by gradient descent with proper early stopping achieve the minimax rate, provided that the regression function lies in the reproducing kernel Hilbert space (RKHS) associated with the corresponding NTK. Finally, we illustrate that the overfitted multilayer wide neural networks can not generalize well on $\mathbb S^{d}$. We believe our technical contributions in determining the eigenvalue decay rate of NTK on $\mathbb R^{d}$ might be of independent interests.

翻译：本文研究定义在有界域$\mathcal X \subset \mathbb R^{d}$上的深度宽前馈ReLU神经网络的泛化能力。我们首先证明神经网络的泛化能力可由对应的深度神经正切核（NTK）回归的泛化能力完全刻画。随后，我们探究深度NTK的谱性质，并证明深度NTK在$\mathcal{X}$上正定，且其特征值衰减率为$(d+1)/d$。借助核回归领域已建立的理论，我们得出结论：若回归函数位于与对应NTK相关联的再生核希尔伯特空间（RKHS）中，则通过梯度下降结合适当早停训练的多层宽神经网络可实现极小极大速率。最后，我们论证过拟合的多层宽神经网络在$\mathbb S^{d}$上无法良好泛化。我们相信，在确定$\mathbb R^{d}$上NTK特征值衰减率方面的技术贡献可能具有独立意义。