Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent neural networks (RNNs) trained to path-integrate. Replay in these networks has been described as Langevin sampling, but new modifiers of noisy RNN replay have surpassed this description. We re-examine noisy RNN replay as sampling to understand or improve it in three ways: (1) Under simple assumptions, we prove that the gradients replay activity should follow are time-varying and difficult to estimate, but readily motivate the use of hidden state leakage in RNNs for replay. (2) We confirm that hidden state adaptation (negative feedback) encourages exploration in replay, but show that it incurs non-Markov sampling that also slows replay. (3) We propose the first model of temporally compressed replay in noisy path-integrating RNNs through hidden state momentum, connect it to underdamped Langevin sampling, and show that, together with adaptation, it counters slowness while maintaining exploration. We verify our findings via path-integration of 2D triangular and T-maze paths and of high-dimensional paths of synthetic rat place cell activity.

翻译：生物神经网络（如海马体）能够内部生成类似于刺激驱动活动的“回放”现象。近期关于回放的计算模型采用经训练实现路径积分的噪声循环神经网络（RNN）。这些网络中的回放曾被描述为朗之万采样，但噪声RNN回放的新型调控机制已超越该描述框架。我们通过采样视角重新审视噪声RNN回放，从三个方面深化理解并改进其性能：（1）在简单假设下，我们证明回放活动应遵循的梯度具有时变特性且难以估计，但这直接启发了在RNN中采用隐藏状态泄漏机制以促进回放。（2）我们证实隐藏状态自适应（负反馈）能增强回放中的探索行为，但同时证明这会引发非马尔可夫采样并减缓回放速度。（3）我们首次通过隐藏状态动量机制，提出噪声路径积分RNN中时间压缩回放的模型，将其与欠阻尼朗之万采样建立理论关联，并证明该机制与自适应结合能在保持探索性的同时克服速度迟滞问题。我们通过二维三角形路径、T型迷宫路径以及合成大鼠位置细胞活动的高维路径积分实验验证了上述发现。