Understanding the intricate operations of Recurrent Neural Networks (RNNs) mechanistically is pivotal for advancing their capabilities and applications. In this pursuit, we propose the Episodic Memory Theory (EMT), illustrating that RNNs can be conceptualized as discrete-time analogs of the recently proposed General Sequential Episodic Memory Model. To substantiate EMT, we introduce a novel set of algorithmic tasks tailored to probe the variable binding behavior in RNNs. Utilizing the EMT, we formulate a mathematically rigorous circuit that facilitates variable binding in these tasks. Our empirical investigations reveal that trained RNNs consistently converge to the variable binding circuit, thus indicating universality in the dynamics of RNNs. Building on these findings, we devise an algorithm to define a privileged basis, which reveals hidden neurons instrumental in the temporal storage and composition of variables, a mechanism vital for the successful generalization in these tasks. We show that the privileged basis enhances the interpretability of the learned parameters and hidden states of RNNs. Our work represents a step toward demystifying the internal mechanisms of RNNs and, for computational neuroscience, serves to bridge the gap between artificial neural networks and neural memory models.

翻译：从机制上理解循环神经网络（RNN）的复杂运算对提升其能力与应用至关重要。为此，我们提出情景记忆理论（EMT），阐明RNN可被视作近期提出的一般序列情景记忆模型的离散时间类比。为验证EMT，我们设计了一组新型算法任务，专门用于探究RNN中的变量绑定行为。利用EMT，我们构建了数学上严谨的电路结构，以支持这些任务中的变量绑定。实验研究表明，经过训练的RNN始终收敛至该变量绑定电路，揭示了RNN动力学的普遍性。基于此发现，我们设计了一种算法来定义特权基，由此揭示负责变量时序存储与组合的关键隐藏神经元——这一机制对这些任务的成功泛化至关重要。研究表明，特权基提升了RNN学习参数与隐藏状态的可解释性。本工作为揭示RNN内部机制迈出关键一步，同时在计算神经科学领域架起了人工神经网络与神经记忆模型之间的桥梁。