Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns. While AI is able to reproduce some of these capabilities, deep learning algorithms remain largely at odds with our current understanding of brain circuitry and dynamics. This is prominently the case for backpropagation through time (BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general formalism to approximate BPTT in a controlled, biologically plausible manner. Our approach builds on, unifies and extends several previous approaches to local, time-continuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action. Our starting point is a prospective energy function of neuronal states, from which we calculate real-time error dynamics for time-continuous neuronal networks. In the general case, this provides a simple and straightforward derivation of the adjoint method result for neuronal networks, the time-continuous equivalent to BPTT. With a few modifications, we can turn this into a fully local (in space and time) set of equations for neuron and synapse dynamics. Our theory provides a rigorous framework for spatiotemporal deep learning in the brain, while simultaneously suggesting a blueprint for physical circuits capable of carrying out these computations. These results reframe and extend the recently proposed Generalized Latent Equilibrium (GLE) model.

翻译：大脑在识别和生成复杂时空模式方面的能力依然无与伦比。尽管人工智能能够复现其中部分能力，但深度学习算法在很大程度上仍与我们当前对大脑回路和动力学的理解相悖。这一点在用于学习复杂时间依赖性的首选算法——随时间反向传播（BPTT）上尤为突出。本工作提出了一种通用形式体系，以可控且生物学合理的方式近似BPTT。我们的方法基于能量守恒和极值作用原理，整合、统一并拓展了先前几种关于局部、时间连续、无相位的时空信用分配的方案。我们的起点是神经元状态的前瞻性能量函数，据此我们推导出时间连续神经元网络的实时误差动力学。在一般情况下，这为神经元网络的伴随方法结果（即BPTT的时间连续等价形式）提供了简洁直接的推导。通过若干修正，我们可以将其转化为一套完全局部（在空间和时间上）的神经元与突触动力学方程组。我们的理论为大脑中的时空深度学习提供了严谨框架，同时为能够执行这些计算的物理回路提出了设计蓝图。这些成果重构并拓展了近期提出的广义潜平衡（GLE）模型。