Spiking neural networks (SNNs) are a bio-inspired alternative to conventional real-valued deep learning models, with the potential for substantially higher energy efficiency. Interest in SNNs has recently exploded due to a major breakthrough: surrogate gradient learning (SGL), which allows training SNNs with backpropagation, strongly outperforming other approaches. In SNNs, each synapse is characterized not only by a weight but also by a transmission delay. While theoretical works have long suggested that trainable delays significantly enhance expressivity, practical methods for learning them have only recently emerged. Here, we introduce ``DelRec'', the first SGL-based method to train axonal or synaptic delays in recurrent spiking layers, compatible with any spiking neuron model. DelRec leverages a differentiable interpolation technique to handle non-integer delays with well-defined gradients at training time. We show that SNNs with trainable recurrent delays outperform feedforward ones, leading to new state-of-the-art (SOTA) on two challenging temporal datasets (Spiking Speech Command, an audio dataset, and Permuted Sequential MNIST, a vision one), and match the SOTA on the now saturated Spiking Heidelberg Digit dataset using only vanilla Leaky-Integrate-and-Fire neurons with stateless (instantaneous) synapses. Our results demonstrate that recurrent delays are critical for temporal processing in SNNs and can be effectively optimized with DelRec, paving the way for efficient deployment on neuromorphic hardware with programmable delays. Our code is available at https://github.com/alexmaxad/DelRec.

翻译：脉冲神经网络（SNNs）是一种受生物启发的、替代传统实值深度学习模型的方案，具有显著提高能效的潜力。由于一项重大突破——替代梯度学习（SGL），使得能够通过反向传播训练SNNs并大幅超越其他方法，近期对SNNs的研究兴趣激增。在SNNs中，每个突触不仅由权重表征，还由一个传输延迟表征。尽管理论工作长期以来都表明可训练的延迟能显著增强表达能力，但学习延迟的实用方法直到最近才出现。本文提出了“DelRec”，这是首个基于SGL、用于训练循环脉冲层中轴突或突触延迟的方法，兼容任何脉冲神经元模型。DelRec利用一种可微分插值技术来处理非整数延迟，并在训练时提供定义良好的梯度。我们证明，具有可训练循环延迟的SNNs优于前馈网络，在两个具有挑战性的时序数据集（Spiking Speech Command，一个音频数据集；以及Permuted Sequential MNIST，一个视觉数据集）上取得了新的最先进（SOTA）性能，并且在现已饱和的Spiking Heidelberg Digit数据集上，仅使用具有无状态（瞬时）突触的标准Leaky-Integrate-and-Fire神经元就匹配了SOTA。我们的结果表明，循环延迟对于SNNs中的时序处理至关重要，并且可以通过DelRec进行有效优化，为在具有可编程延迟的神经形态硬件上高效部署铺平了道路。我们的代码可在 https://github.com/alexmaxad/DelRec 获取。