Inter-neuron communication delays are ubiquitous in physically realized neural networks such as biological neural circuits and neuromorphic hardware. These delays have significant and often disruptive consequences on network dynamics during training and inference. It is therefore essential that communication delays be accounted for, both in computational models of biological neural networks and in large-scale neuromorphic systems. Nonetheless, communication delays have yet to be comprehensively addressed in either domain. In this paper, we first show that delays prevent state-of-the-art continuous-time neural networks called Latent Equilibrium (LE) networks from learning even simple tasks despite significant overparameterization. We then propose to compensate for communication delays by predicting future signals based on currently available ones. This conceptually straightforward approach, which we call prospective messaging (PM), uses only neuron-local information, and is flexible in terms of memory and computation requirements. We demonstrate that incorporating PM into delayed LE networks prevents reaction lags, and facilitates successful learning on Fourier synthesis and autoregressive video prediction tasks.

翻译：神经元间通信延迟在物理实现的神经网络（如生物神经回路与神经形态硬件）中普遍存在。这些延迟在训练与推理期间对网络动力学具有显著且通常具有破坏性的影响。因此，在生物神经网络的计算模型与大规模神经形态系统中，都必须考虑通信延迟。然而，通信延迟在这两个领域均尚未得到全面解决。本文首先表明，延迟会阻碍称为潜在均衡（LE）网络的最先进连续时间神经网络学习即使是简单的任务，尽管存在显著的过参数化。随后，我们提出通过基于当前可用信号预测未来信号来补偿通信延迟。这种概念上直接的方法，我们称之为前瞻性消息传递（PM），仅使用神经元局部信息，并且在内存和计算需求方面具有灵活性。我们证明，将PM整合到具有延迟的LE网络中能够防止反应滞后，并促进在傅里叶合成与自回归视频预测任务上的成功学习。