Neural networks are high-dimensional nonlinear dynamical systems that process information through the coordinated activity of many connected units. Understanding how biological and machine-learning networks function and learn requires knowledge of the structure of this coordinated activity, information contained, for example, in cross covariances between units. Self-consistent dynamical mean field theory (DMFT) has elucidated several features of random neural networks -- in particular, that they can generate chaotic activity -- however, a calculation of cross covariances using this approach has not been provided. Here, we calculate cross covariances self-consistently via a two-site cavity DMFT. We use this theory to probe spatiotemporal features of activity coordination in a classic random-network model with independent and identically distributed (i.i.d.) couplings, showing an extensive but fractionally low effective dimension of activity and a long population-level timescale. Our formulae apply to a wide range of single-unit dynamics and generalize to non-i.i.d. couplings. As an example of the latter, we analyze the case of partially symmetric couplings.

翻译：神经网络是高维非线性动力系统，通过大量连接单元的协调活动处理信息。理解生物与机器学习网络的功能和学习机制，需要掌握这种协调活动的结构——例如体现在单元间互协方差中的信息。自洽动力学平均场理论（DMFT）已揭示随机神经网络的若干特征，尤其是其能产生混沌活动，但尚未利用该方法给出互协方差的计算。本文通过双位点空穴腔DMFT自洽计算互协方差，并运用该理论探究经典随机网络模型（具有独立同分布耦合）中活动协调的时空特征，揭示出活动具有广延但分数化的低有效维度以及长群体级时间尺度。我们的公式适用于广泛的单单元动力学，并可推广至非独立同分布耦合。作为后者示例，我们分析了部分对称耦合情形。