The remarkable generalization performance of large-scale models has been challenging the conventional wisdom of the statistical learning theory. Although recent theoretical studies have shed light on this behavior in linear models and nonlinear classifiers, a comprehensive understanding of overparameterization in nonlinear regression models is still lacking. This study explores the predictive properties of overparameterized nonlinear regression within the Bayesian framework, extending the methodology of the adaptive prior considering the intrinsic spectral structure of the data. Posterior contraction is established for generalized linear and single-neuron models with Lipschitz continuous activation functions, demonstrating the consistency in the predictions of the proposed approach. Moreover, the Bayesian framework enables uncertainty estimation of the predictions. The proposed method was validated via numerical simulations and a real data application, showing its ability to achieve accurate predictions and reliable uncertainty estimates. This work provides a theoretical understanding of the advantages of overparameterization and a principled Bayesian approach to large nonlinear models.

翻译：大规模模型卓越的泛化性能不断挑战着统计学习理论的传统认知。尽管近期理论研究已在线性模型与非线性分类器领域揭示了该现象的内在机制，但对于非线性回归模型中过参数化行为的系统认识仍然不足。本研究在贝叶斯框架下探索过参数化非线性回归的预测特性，通过扩展自适应先验方法以融合数据内在谱结构。针对具有Lipschitz连续激活函数的广义线性模型与单神经元模型，建立了后验收缩的理论保证，证明了所提方法在预测层面的一致性。此外，贝叶斯框架能够实现对预测结果的不确定性量化。通过数值模拟与真实数据实验验证，该方法在获得精确预测的同时能够提供可靠的不确定性估计。本研究不仅为过参数化的优势提供了理论解释，更为大规模非线性模型提供了原则性的贝叶斯推断框架。