Inter-neuron communication delays are ubiquitous in physically realized neural networks such as biological neural circuits and neuromorphic hardware. These delays have significant and often disruptive consequences on network dynamics during training and inference. It is therefore essential that communication delays be accounted for, both in computational models of biological neural networks and in large-scale neuromorphic systems. Nonetheless, communication delays have yet to be comprehensively addressed in either domain. In this paper, we first show that delays prevent state-of-the-art continuous-time neural networks called Latent Equilibrium (LE) networks from learning even simple tasks despite significant overparameterization. We then propose to compensate for communication delays by predicting future signals based on currently available ones. This conceptually straightforward approach, which we call prospective messaging (PM), uses only neuron-local information, and is flexible in terms of memory and computation requirements. We demonstrate that incorporating PM into delayed LE networks prevents reaction lags, and facilitates successful learning on Fourier synthesis and autoregressive video prediction tasks.

翻译：神经元间通信延迟在物理实现的神经网络（如生物神经回路和神经形态硬件）中普遍存在。这些延迟在训练和推理过程中对网络动力学产生显著且通常是破坏性的影响。因此，在生物神经网络的计算模型和大规模神经形态系统中，都必须考虑通信延迟。然而，在这两个领域中，通信延迟尚未得到全面解决。本文首先证明，延迟会阻碍称为潜在平衡（LE）网络的最先进连续时间神经网络学习简单任务，即使网络存在显著过参数化。随后，我们提出通过基于当前可用信号预测未来信号来补偿通信延迟。这种概念上直接的方法——我们称之为前瞻性消息传递（PM）——仅使用神经元局部信息，并且在内存和计算需求方面具有灵活性。我们证明，将PM整合到延迟的LE网络中能够防止反应滞后，并促进在傅里叶合成和自回归视频预测任务上的成功学习。