Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Compared to conventional deep Artificial Neural Networks (ANNs), SNNs exhibit superior efficiency and capability to process temporal information. However, it remains a challenge to train SNNs due to their undifferentiable spiking mechanism. The surrogate gradients method is commonly used to train SNNs, but often comes with an accuracy disadvantage over ANNs counterpart. We link the degraded accuracy to the vanishing of gradient on the temporal dimension through the analytical and experimental study of the training process of Leaky Integrate-and-Fire (LIF) Neuron-based SNNs. Moreover, we propose the Complementary Leaky Integrate-and-Fire (CLIF) Neuron. CLIF creates extra paths to facilitate the backpropagation in computing temporal gradient while keeping binary output. CLIF is hyperparameter-free and features broad applicability. Extensive experiments on a variety of datasets demonstrate CLIF's clear performance advantage over other neuron models. Furthermore, the CLIF's performance even slightly surpasses superior ANNs with identical network structure and training conditions. The code is available at https://github.com/HuuYuLong/Complementary-LIF.

翻译：脉冲神经网络（SNNs）是一种前景广阔的类脑高能效模型。与传统的深度人工神经网络（ANNs）相比，SNNs在处理时序信息方面展现出更高的效率和能力。然而，由于其不可微的脉冲发放机制，训练SNNs仍具挑战性。代理梯度法通常用于训练SNNs，但其精度往往不及对应的ANNs。通过对基于漏积分发放（LIF）神经元的SNNs训练过程进行理论分析和实验研究，我们将精度下降归因于时序维度上的梯度消失。此外，我们提出了互补漏积分发放（CLIF）神经元。CLIF通过创建额外路径来促进时序梯度计算中的反向传播，同时保持二值输出。CLIF无需超参数调整且具有广泛适用性。在多种数据集上的大量实验表明，CLIF相较于其他神经元模型具有明显的性能优势。此外，在相同网络结构和训练条件下，CLIF的性能甚至略微优于先进的ANNs。代码发布于 https://github.com/HuuYuLong/Complementary-LIF。