Spiking Neural Networks (SNNs) are a promising research direction for building power-efficient information processing systems, especially for temporal tasks such as speech recognition. In SNNs, delays refer to the time needed for one spike to travel from one neuron to another. These delays matter because they influence the spike arrival times, and it is well-known that spiking neurons respond more strongly to coincident input spikes. More formally, it has been shown theoretically that plastic delays greatly increase the expressivity in SNNs. Yet, efficient algorithms to learn these delays have been lacking. Here, we propose a new discrete-time algorithm that addresses this issue in deep feedforward SNNs using backpropagation, in an offline manner. To simulate delays between consecutive layers, we use 1D convolutions across time. The kernels contain only a few non-zero weights - one per synapse - whose positions correspond to the delays. These positions are learned together with the weights using the recently proposed Dilated Convolution with Learnable Spacings (DCLS). We evaluated our method on three datasets: the Spiking Heidelberg Dataset (SHD), the Spiking Speech Commands (SSC) and its non-spiking version Google Speech Commands v0.02 (GSC) benchmarks, which require detecting temporal patterns. We used feedforward SNNs with two or three hidden fully connected layers, and vanilla leaky integrate-and-fire neurons. We showed that fixed random delays help and that learning them helps even more. Furthermore, our method outperformed the state-of-the-art in the three datasets without using recurrent connections and with substantially fewer parameters. Our work demonstrates the potential of delay learning in developing accurate and precise models for temporal data processing. Our code is based on PyTorch / SpikingJelly and available at: https://github.com/Thvnvtos/SNN-delays

翻译：脉冲神经网络（SNN）是构建高能效信息处理系统的有前景研究方向，尤其适用于语音识别等时序任务。在SNN中，延迟是指单个脉冲从一个神经元传播到另一个神经元所需的时间。这些延迟之所以重要，是因为它们会影响脉冲到达时间，而众所周知，脉冲神经元对同时到达的输入脉冲响应更强。更严格的理论证明表明，可塑性延迟能大幅提升SNN的表达能力。然而，目前尚缺乏高效学习这些延迟的算法。本文提出了一种新的离散时间算法，通过反向传播以离线方式解决深度前馈SNN中的延迟学习问题。为模拟连续层间的延迟，我们采用跨时间的一维卷积，其卷积核仅包含少数非零权重（每个突触对应一个），这些权重的位置对应延迟值。我们结合最近提出的可学习间隔膨胀卷积（DCLS）技术，同步学习权重位置与权重值。我们在三个需要检测时序模式的数据集上评估了该方法：脉冲海德堡数据集（SHD）、脉冲语音指令集（SSC）及其非脉冲版本Google语音指令集v0.02（GSC）。模型采用含两到三个隐藏全连接层的前馈SNN，神经元类型为经典泄漏积分点火模型。实验表明，固定随机延迟有效，而学习延迟效果更优。此外，该方法在不使用循环连接且参数显著减少的情况下，在三个数据集上均超越了当前最优水平。本研究揭示了延迟学习在构建精确时序数据处理模型中的巨大潜力。代码基于PyTorch/SpikingJelly开发，开源地址：https://github.com/Thvnvtos/SNN-delays