As an emerging network model, spiking neural networks (SNNs) have aroused significant research attentions in recent years. However, the energy-efficient binary spikes do not augur well with gradient descent-based training approaches. Surrogate gradient (SG) strategy is investigated and applied to circumvent this issue and train SNNs from scratch. Due to the lack of well-recognized SG selection rule, most SGs are chosen intuitively. We propose the parametric surrogate gradient (PSG) method to iteratively update SG and eventually determine an optimal surrogate gradient parameter, which calibrates the shape of candidate SGs. In SNNs, neural potential distribution tends to deviate unpredictably due to quantization error. We evaluate such potential shift and propose methodology for potential distribution adjustment (PDA) to minimize the loss of undesired pre-activations. Experimental results demonstrate that the proposed methods can be readily integrated with backpropagation through time (BPTT) algorithm and help modulated SNNs to achieve state-of-the-art performance on both static and dynamic dataset with fewer timesteps.

翻译：作为一种新兴的网络模型，脉冲神经网络（SNNs）近年来引起了广泛的研究关注。然而，能量高效的二元脉冲与基于梯度下降的训练方法并不兼容。为克服这一问题并从头训练SNNs，研究者们探索并应用了代理梯度（SG）策略。由于缺乏公认的SG选择规则，大多数SG是凭直觉选取的。我们提出参数化代理梯度（PSG）方法，通过迭代更新SG，最终确定最优的代理梯度参数，从而校准候选SG的形状。在SNNs中，由于量化误差，神经电位分布往往会出现不可预测的偏移。我们评估了这种电位漂移，并提出了膜电位分布调整（PDA）方法，以最小化不良预激活造成的损失。实验结果表明，所提方法可轻松与随时间反向传播（BPTT）算法集成，并帮助调制的SNNs在更少的时间步下，在静态和动态数据集上均达到最先进的性能。