The statistics of correlations are central quantities characterizing the collective dynamics of recurrent neural networks. We derive exact expressions for the statistics of correlations of nonlinear recurrent networks in the limit of a large number N of neurons, including systematic 1/N corrections, in the regime of Gaussian quenched disorder. Our approach uses a path-integral representation of the network stochastic dynamics, which reduces the description to a few collective variables and enables efficient computation. This generalizes previous results on linear networks to include a wide family of nonlinear activation functions, which enter as interaction terms in the path integral. These interactions can resolve the instability of the linear theory and yield a strictly positive participation dimension. We present explicit results for power-law activations, revealing scaling behavior controlled by the network coupling. In addition, we introduce a class of activation functions based on Pade approximants and provide analytic predictions for their correlation statistics. Numerical simulations confirm our theoretical results with excellent agreement. We also compare with previous works that have studied the complementary case with annealed disorder, and based on this we propose a new self-consistent equation for the more general case of colored noise.

翻译：相关性统计是描述循环神经网络集体动力学的核心量。我们推导了在大量神经元N的极限下，非线性循环网络相关性统计的精确表达式，包括系统性的1/N修正，该表达式适用于高斯淬火无序体系。我们的方法采用网络随机动力学的路径积分表示，将描述简化为少数集体变量，并实现了高效计算。这推广了先前关于线性网络的结果，涵盖了广泛的非线性激活函数族，这些函数在路径积分中作为相互作用项出现。这些相互作用能够解决线性理论的不稳定性，并产生严格正值的参与维度。我们给出了幂律激活函数的显式结果，揭示了由网络耦合控制的标度行为。此外，我们引入了一类基于Padé近似的激活函数，并提供了其相关性统计的分析预测。数值模拟与我们的理论结果高度吻合。我们还与先前研究退火无序互补情况的文献进行了比较，并据此提出了一个适用于有色噪声更一般情况的新自洽方程。