Understanding neural dynamics is a central topic in machine learning, non-linear physics and neuroscience. However, the dynamics is non-linear, stochastic and particularly non-gradient, i.e., the driving force can not be written as gradient of a potential. These features make analytic studies very challenging. The common tool is to use path integral approach or dynamical mean-field theory, but the drawback is one has to solve the integro-differential or dynamical mean-field equations, which is computationally expensive and has no closed form solutions in general. From the aspect of associated Fokker-Planck equation, the steady state solution is generally unknown. Here, we treat searching for the steady state as an optimization problem, and construct an approximate potential closely related to the speed of the dynamics, and find that searching for the ground state of this potential is equivalent to running a stochastic gradient dynamics. The resultant stationary state follows exactly the canonical Boltzmann measure. Within this framework, the quenched disorder intrinsic in the neural networks can be averaged out by applying the replica method. Our theory reproduces the well-known result of edge-of-chaos, and further the order parameters characterizing the continuous transition are derived, and different scaling behavior with respect to inverse temperature in both sides of the transition is also revealed. Our method opens the door to analytically study the steady state landscape of the deterministic or stochastic high dimensional dynamics.

翻译：理解神经动力学是机器学习、非线性物理和神经科学领域的核心课题。然而，该动力系统具有非线性、随机性，特别是非梯度性（即驱动力无法表示为势函数的梯度），这些特性使得解析研究极具挑战性。常用方法包括路径积分方法或动态平均场理论，但缺点是需要求解积分微分方程或动态平均场方程，这不仅计算成本高昂，且通常不存在闭式解。从关联的福克-普朗克方程角度看，其稳态解通常未知。本文将从求解稳态问题转化为优化问题，构造一个与动力学速度密切相关的近似势能，发现搜索该势能基态等价于运行随机梯度动力学，所得的稳态严格遵循正则玻尔兹曼测度。在该框架下，神经网络中固有的淬火无序可通过副本方法平均化。本文理论不仅重现了混沌边缘的经典结论，还推导了表征连续相变的序参量，进而揭示了相变两侧关于逆温度的不同标度行为。该方法为解析研究确定性或随机高维动力学的稳态景观开辟了新途径。