Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, state-of-the-art deep SNNs (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connection. These non-spike computations increase SNNs' power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware, which only supports spike operations. In this paper, we propose a hardware-friendly spike-driven residual learning architecture for SNNs to avoid non-spike computations. Based on this residual design, we develop Spikingformer, a pure transformer-based spiking neural network. We evaluate Spikingformer on ImageNet, CIFAR10, CIFAR100, CIFAR10-DVS and DVS128 Gesture datasets, and demonstrated that Spikingformer outperforms the state-of-the-art in directly trained pure SNNs as a novel advanced backbone (74.79$\%$ top-1 accuracy on ImageNet, + 1.41$\%$ compared with Spikformer). Furthermore, our experiments verify that Spikingformer effectively avoids non-spike computations and reduces energy consumption by 60.34$\%$ compared with Spikformer on ImageNet. To our best knowledge, this is the first time that a pure event-driven transformer-based SNN has been developed.

翻译：脉冲神经网络（SNN）因其事件驱动的脉冲计算特性，成为人工神经网络极具前景的节能替代方案。然而，当前最先进的深度SNN（包括Spikformer和SEW ResNet）因残差连接结构存在非脉冲计算（整数-浮点乘法）。这些非脉冲计算增加了SNN的功耗，且使其无法部署在仅支持脉冲操作的主流神经形态硬件上。本文提出了一种硬件友好的脉冲驱动残差学习架构以消除非脉冲计算。基于该残差设计，我们开发了Spikingformer——一种纯Transformer类脉冲神经网络。在ImageNet、CIFAR10、CIFAR100、CIFAR10-DVS和DVS128 Gesture数据集上的评估表明，Spikingformer作为新型先进骨干网络，其性能优于现有直接训练的纯SNN方法（ImageNet top-1准确率74.79%，较Spikformer提升1.41%）。此外，实验验证了Spikingformer能有效避免非脉冲计算，相较Spikformer在ImageNet上降低60.34%的能耗。据我们所知，这是首次实现纯事件驱动的Transformer类SNN。