Traveling waves of neural activity have been observed throughout the brain at a diversity of regions and scales; however, their precise computational role is still debated. One physically grounded hypothesis suggests that the cortical sheet may act like a wave-field capable of storing a short-term memory of sequential stimuli through induced waves traveling across the cortical surface. To date, however, the computational implications of this idea have remained hypothetical due to the lack of a simple recurrent neural network architecture capable of exhibiting such waves. In this work, we introduce a model to fill this gap, which we denote the Wave-RNN (wRNN), and demonstrate how both connectivity constraints and initialization play a crucial role in the emergence of wave-like dynamics. We then empirically show how such an architecture indeed efficiently encodes the recent past through a suite of synthetic memory tasks where wRNNs learn faster and perform significantly better than wave-free counterparts. Finally, we explore the implications of this memory storage system on more complex sequence modeling tasks such as sequential image classification and find that wave-based models not only again outperform comparable wave-free RNNs while using significantly fewer parameters, but additionally perform comparably to more complex gated architectures such as LSTMs and GRUs. We conclude with a discussion of the implications of these results for both neuroscience and machine learning.

翻译：神经活动的行波已在全脑不同区域和尺度上被观测到，但其精确的计算作用仍存在争议。一个基于物理学的假说认为，皮层片层可能像一个波场，能够通过跨越皮层表面传播的诱导波来存储序列刺激的短期记忆。然而，迄今为止，由于缺乏能够展现此类波的简单递归神经网络架构，这一观点的计算意义仍仅停留在假设层面。在本工作中，我们引入了一个填补这一空白的模型，命名为Wave-RNN（wRNN），并展示了连接约束和初始化在波状动力学涌现中的关键作用。随后，我们通过一套合成记忆任务实证证明，这种架构确实能高效编码近期历史，其中wRNN的学习速度更快且性能显著优于无波对应模型。最后，我们探究了这种记忆存储系统对更复杂序列建模任务（如序列图像分类）的影响，发现基于波的模型不仅在使用更少参数的情况下再次超越可比较的无波RNN，其性能还堪比更复杂的门控架构（如LSTM和GRU）。我们最后讨论了这些结果对神经科学和机器学习的意义。