Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typically rely on first-order Ordinary Differential Equations (ODEs) to govern neuronal state transitions. This first-order assumption imposes a "memoryless" bottleneck, limiting the model's capacity to capture the complex, long-range dependencies inherent in long-sequence tasks. In this work, we propose LongSpike, a novel SNN framework that integrates fractional-order State-Space Modeling, or f-SSM, from control theory into the spiking domain. By extending traditional integer-order SSMs to the fractional-calculus regime, LongSpike enables the hierarchical integration of neuronal dynamics with long-memory kernels. To mitigate the computational overhead and parallelization challenges typically associated with fractional operators, we leverage a state-space formulation that supports efficient, parallel training. Empirical evaluations on challenging benchmarks, including Long Range Arena (LRA), large-scale WikiText-103, and Speech Commands, demonstrate that LongSpike outperforms state-of-the-art SNNs in accuracy while preserving sparse synaptic computation. The code is available at https://github.com/xinruihe389-commits/LongSpike.
翻译:脉冲神经网络因其生物合理性和处理序列数据时的能效而备受推崇。然而,主流的脉冲神经网络架构通常依赖一阶常微分方程来控制神经元状态跃迁。这种一阶假设导致了“无记忆”瓶颈,限制了模型捕获长序列任务中固有的复杂长程依赖关系的能力。在本工作中,我们提出了LongSpike,一种新颖的脉冲神经网络框架,它将控制论中的分数阶状态空间模型(f-SSM)集成到脉冲域中。通过将传统的整数阶状态空间模型扩展到分数阶微积分领域,LongSpike实现了具有长记忆核的神经元动力学的层次化整合。为了缓解分数阶算子通常带来的计算开销和并行化挑战,我们采用了一种支持高效并行训练的状态空间公式。在包括Long Range Arena、大规模WikiText-103和语音指令在内的挑战性基准测试上的实证评估表明,LongSpike在精度上超越了最先进的脉冲神经网络,同时保持了稀疏突触计算。代码可在https://github.com/xinruihe389-commits/LongSpike获取。