Spiking Neural Networks (SNNs) are a promising research direction for building power-efficient information processing systems, especially for temporal tasks such as speech recognition. In SNNs, delays refer to the time needed for one spike to travel from one neuron to another. These delays matter because they influence the spike arrival times, and it is well-known that spiking neurons respond more strongly to coincident input spikes. More formally, it has been shown theoretically that plastic delays greatly increase the expressivity in SNNs. Yet, efficient algorithms to learn these delays have been lacking. Here, we propose a new discrete-time algorithm that addresses this issue in deep feedforward SNNs using backpropagation, in an offline manner. To simulate delays between consecutive layers, we use 1D convolutions across time. The kernels contain only a few non-zero weights - one per synapse - whose positions correspond to the delays. These positions are learned together with the weights using the recently proposed Dilated Convolution with Learnable Spacings (DCLS). We evaluated our method on the Spiking Heidelberg Dataset (SHD) and the Spiking Speech Commands (SSC) benchmarks, which require detecting temporal patterns. We used feedforward SNNs with two hidden fully connected layers. We showed that fixed random delays help, and that learning them helps even more. Furthermore, our method outperformed the state-of-the-art in both SHD and SSC without using recurrent connections and with substantially fewer parameters. Our work demonstrates the potential of delay learning in developing accurate and precise models for temporal data processing. Our code is based on PyTorch / SpikingJelly and available at: https://github.com/Thvnvtos/SNN-delays

翻译：脉冲神经网络（SNNs）是构建低功耗信息处理系统的一个有前景的研究方向，尤其适用于语音识别等时序任务。在SNNs中，延迟指一个脉冲从一个神经元传播到另一个神经元所需的时间。这些延迟至关重要，因为它们影响脉冲到达时间，且众所周知，脉冲神经元对同步输入脉冲的响应更强。更形式化地，理论研究表明可塑性延迟能显著提高SNNs的表达能力。然而，目前仍缺乏高效学习这些延迟的算法。本文提出了一种新的离散时间算法，通过反向传播以离线方式解决深度前馈SNNs中的这一问题。为模拟相邻层之间的延迟，我们使用跨时间的一维卷积。卷积核仅包含少量非零权重——每个突触对应一个——其位置对应延迟值。这些位置与权重一起，通过近期提出的可学习间距扩张卷积（DCLS）进行学习。我们在脉冲海德堡数据集（SHD）和脉冲语音指令（SSC）基准上评估了该方法，这两项任务均需检测时间模式。我们采用具有两个隐藏全连接层的前馈SNNs。实验表明，固定随机延迟有所助益，而学习延迟效果更佳。此外，我们的方法在不使用循环连接且参数大幅减少的情况下，在SHD和SSC上均超越了现有最优水平。本工作展示了延迟学习在开发精准时序数据处理模型方面的潜力。代码基于PyTorch/SpikingJelly，开源地址：https://github.com/Thvnvtos/SNN-delays