Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, state-of-the-art deep SNNs (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connection. These non-spike computations increase SNNs' power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware, which only supports spike operations. In this paper, we propose a hardware-friendly spike-driven residual learning architecture for SNNs to avoid non-spike computations. Based on this residual design, we develop Spikingformer, a pure transformer-based spiking neural network. We evaluate Spikingformer on ImageNet, CIFAR10, CIFAR100, CIFAR10-DVS and DVS128 Gesture datasets, and demonstrate that Spikingformer outperforms the state-of-the-art in directly trained pure SNNs as a novel advanced backbone (75.85$\%$ top-1 accuracy on ImageNet, + 1.04$\%$ compared with Spikformer). Furthermore, our experiments verify that Spikingformer effectively avoids non-spike computations and significantly reduces energy consumption by 57.34$\%$ compared with Spikformer on ImageNet. To our best knowledge, this is the first time that a pure event-driven transformer-based SNN has been developed.

翻译：脉冲神经网络因其事件驱动的脉冲计算特性，成为人工神经网络的一种有前景的节能替代方案。然而，当前先进的深度脉冲神经网络（包括Spikformer和SEW ResNet）因其残差连接结构而存在非脉冲计算（整数-浮点数乘法）。这些非脉冲计算增加了脉冲神经网络的功耗，使其无法部署在仅支持脉冲操作的主流神经形态硬件上。本文提出一种硬件友好的脉冲驱动残差学习架构，以规避脉冲神经网络中的非脉冲计算。基于该残差设计，我们开发了Spikingformer——一种纯Transformer架构的脉冲神经网络。我们在ImageNet、CIFAR10、CIFAR100、CIFAR10-DVS和DVS128 Gesture数据集上评估Spikingformer，证明其作为新型先进骨干网络在直接训练的纯脉冲神经网络中超越现有最优水平（ImageNet Top-1准确率75.85%，较Spikformer提升1.04%）。此外，实验结果证实Spikingformer有效避免了非脉冲计算，在ImageNet上较Spikformer显著降低57.34%的能量消耗。据我们所知，这是首次实现完全基于事件驱动的Transformer架构脉冲神经网络。