We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime. As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of ODEs. Our framework applies to a broad range of network architectures, learning dynamics and network initializations. Central to the framework is the new idea of a \textit{neuronal embedding}, which comprises of a non-evolving probability space that allows to embed neural networks of arbitrary widths. Using our framework, we prove several properties of large-width multilayer neural networks. Firstly we show that independent and identically distributed initializations cause strong degeneracy effects on the network's learning trajectory when the network's depth is at least four. Secondly we obtain several global convergence guarantees for feedforward multilayer networks under a number of different setups. These include two-layer and three-layer networks with independent and identically distributed initializations, and multilayer networks of arbitrary depths with a special type of correlated initializations that is motivated by the new concept of \textit{bidirectional diversity}. Unlike previous works that rely on convexity, our results admit non-convex losses and hinge on a certain universal approximation property, which is a distinctive feature of infinite-width neural networks and is shown to hold throughout the training process. Aside from being the first known results for global convergence of multilayer networks in the mean field regime, they demonstrate flexibility of our framework and incorporate several new ideas and insights that depart from the conventional convex optimization wisdom.

翻译：本文针对平均场机制下的多层神经网络建立了数学严格的理论框架。随着网络宽度增加，网络的学习轨迹能够被一个具有动态非线性的有意义极限（即平均场极限）精确描述，该极限由常微分方程组刻画。该框架适用于广泛的网络架构、学习动力学和网络初始化方式。其核心创新在于提出了"神经元嵌入"概念——通过构建一个非演化的概率空间，实现对任意宽度神经网络的嵌入。基于该框架，我们证明了多层大宽度网络的若干性质：首先，当网络深度至少为四层时，独立同分布初始化会对学习轨迹产生强退化效应；其次，我们在多种设置下获得了前馈多层网络的全局收敛保证，包括采用独立同分布初始化的两层和三层网络，以及具有特殊相关初始化（受新型"双向多样性"概念启发）的任意深度多层网络。与依赖凸性的既有研究不同，我们的结果允许非凸损失函数，且其证明依赖于无限宽度神经网络的独特特性——通用逼近性在训练过程中的持续性。作为平均场机制下多层网络全局收敛的首批已知结果，这些成果既彰显了框架的灵活性，也引入了突破传统凸优化范式的创新思想与洞见。