We establish explicit dynamics for neural networks whose training objective has a regularising term that constrains the parameters to remain close to their initial value. This keeps the network in a lazy training regime, where the dynamics can be linearised around the initialisation. The standard neural tangent kernel (NTK) governs the evolution during the training in the infinite-width limit, although the regularisation yields an additional term appears in the differential equation describing the dynamics. This setting provides an appropriate framework to study the evolution of wide networks trained to optimise generalisation objectives such as PAC-Bayes bounds, and hence potentially contribute to a deeper theoretical understanding of such networks.

翻译：我们建立了训练目标中包含正则化项（约束参数保持接近初始值）的神经网络的显式动力学。该正则化使网络处于懒训练机制，此时动力学可在初始化附近线性化。在无限宽极限下，标准神经正切核（NTK）支配训练过程中的演化，但正则化在描述动力学的微分方程中引入了一个附加项。这一设置为研究为优化泛化目标（如PAC-Bayes界）而训练的宽网络演化提供了恰当框架，从而可能有助于深化对此类网络的理论理解。