Neural dynamical systems with stable attractor structures, such as point attractors and continuous attractors, are hypothesized to underlie meaningful temporal behavior that requires working memory. However, working memory may not support useful learning signals necessary to adapt to changes in the temporal structure of the environment. We show that in addition to the continuous attractors that are widely implicated, periodic and quasi-periodic attractors can also support learning arbitrarily long temporal relationships. Unlike the continuous attractors that suffer from the fine-tuning problem, the less explored quasi-periodic attractors are uniquely qualified for learning to produce temporally structured behavior. Our theory has broad implications for the design of artificial learning systems and makes predictions about observable signatures of biological neural dynamics that can support temporal dependence learning and working memory. Based on our theory, we developed a new initialization scheme for artificial recurrent neural networks that outperforms standard methods for tasks that require learning temporal dynamics. Moreover, we propose a robust recurrent memory mechanism for integrating and maintaining head direction without a ring attractor.

翻译：具有稳定吸引子结构的神经动力系统，如点吸引子和连续吸引子，被认为是支撑需要工作记忆的有意义时间行为的基础。然而，工作记忆可能无法提供适应环境时间结构变化所需的有用学习信号。我们证明，除了被广泛研究的连续吸引子外，周期吸引子和准周期吸引子同样能够支持对任意长时程时间关系的学习。与存在精细调节问题的连续吸引子不同，较少被探索的准周期吸引子在产生时间结构化行为的学习过程中具有独特优势。该理论对人工学习系统的设计具有广泛启示，并可预测生物神经动力学中支持时间依赖性学习与工作记忆的可观测特征。基于该理论，我们为人工循环神经网络开发了一种新的初始化方案，在处理需要学习时间动态的任务中优于标准方法。此外，我们提出了一种无需环形吸引子的鲁棒性循环记忆机制，用于整合与维持头部朝向信息。