How can we tell whether two neural networks utilize the same internal processes for a particular computation? This question is pertinent for multiple subfields of neuroscience and machine learning, including neuroAI, mechanistic interpretability, and brain-machine interfaces. Standard approaches for comparing neural networks focus on the spatial geometry of latent states. Yet in recurrent networks, computations are implemented at the level of dynamics, and two networks performing the same computation with equivalent dynamics need not exhibit the same geometry. To bridge this gap, we introduce a novel similarity metric that compares two systems at the level of their dynamics, called Dynamical Similarity Analysis (DSA). Our method incorporates two components: Using recent advances in data-driven dynamical systems theory, we learn a high-dimensional linear system that accurately captures core features of the original nonlinear dynamics. Next, we compare different systems passed through this embedding using a novel extension of Procrustes Analysis that accounts for how vector fields change under orthogonal transformation. In four case studies, we demonstrate that our method disentangles conjugate and non-conjugate recurrent neural networks (RNNs), while geometric methods fall short. We additionally show that our method can distinguish learning rules in an unsupervised manner. Our method opens the door to comparative analyses of the essential temporal structure of computation in neural circuits.

翻译：如何判断两个神经网络在特定计算中是否使用了相同的内部过程？这个问题与神经科学和机器学习的多个子领域相关，包括神经人工智能（neuroAI）、机制可解释性和脑机接口。比较神经网络的标准方法侧重于潜在状态的空间几何结构。然而，在循环网络中，计算是在动力学层面实现的，执行相同计算且具有等效动力学的两个网络未必表现出相同的几何结构。为了弥合这一差距，我们引入了一种新颖的相似性度量，从动力学层面比较两个系统，称为动力学相似性分析（Dynamical Similarity Analysis，DSA）。我们的方法包含两个部分：利用数据驱动的动力学系统理论的最新进展，我们学习一个高维线性系统，准确捕捉原始非线性动力学的核心特征。接着，我们通过一种新颖的普罗克鲁斯特分析（Procrustes Analysis）扩展来比较通过这种嵌入处理后的不同系统，该扩展考虑了向量场在正交变换下的变化。在四个案例研究中，我们证明了我们的方法能够区分共轭和非共轭的循环神经网络（RNN），而几何方法则难以做到。我们进一步展示，我们的方法可以无监督地区分学习规则。我们的方法为比较分析神经回路中计算的基本时间结构打开了大门。