The combination of Spiking Neural Networks (SNNs) and Transformers has attracted significant attention due to their potential for high energy efficiency and high-performance nature. However, existing works on this topic typically rely on direct training, which can lead to suboptimal performance. To address this issue, we propose to leverage the benefits of the ANN-to-SNN conversion method to combine SNNs and Transformers, resulting in significantly improved performance over existing state-of-the-art SNN models. Furthermore, inspired by the quantal synaptic failures observed in the nervous system, which reduces the number of spikes transmitted across synapses, we introduce a novel Masked Spiking Transformer (MST) framework that incorporates a Random Spike Masking (RSM) method to prune redundant spikes and reduce energy consumption without sacrificing performance. Our experimental results demonstrate that the proposed MST model achieves a significant reduction of 26.8% in power consumption when the masking ratio is 75% while maintaining the same level of performance as the unmasked model.

翻译：脉冲神经网络（SNNs）与Transformer的结合因兼具高能效和高性能的潜力而备受关注。然而，现有相关研究通常依赖直接训练方法，这可能导致性能欠佳。为解决该问题，我们提出利用ANN到SNN转换方法的优势来融合SNNs与Transformer，使模型性能显著超越现有最先进的SNN模型。此外，受神经系统中可减少突触脉冲传输数量的量子化突触失效机制启发，我们引入了一种新颖的掩码脉冲Transformer（MST）框架，该框架采用随机脉冲掩码（RSM）方法修剪冗余脉冲，在保证性能不受损的前提下降低能耗。实验结果表明，所提出的MST模型在掩码率为75%时，可在保持与未掩码模型同等性能水平的情况下，使功耗显著降低26.8%。