Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to the traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent the problem surrogate method uses a differentiable approximation of the Heaviside in the backward pass, while the forward pass uses the Heaviside as the spiking function. We propose to use the zeroth order technique at the neuron level to resolve this dichotomy and use it within the automatic differentiation tool. As a result, we establish a theoretical connection between the proposed local zeroth-order technique and the existing surrogate methods and vice-versa. The proposed method naturally lends itself to energy-efficient training of SNNs on GPUs. Experimental results with neuromorphic datasets show that such implementation requires less than 1 percent neurons to be active in the backward pass, resulting in a 100x speed-up in the backward computation time. Our method offers better generalization compared to the state-of-the-art energy-efficient technique while maintaining similar efficiency.

翻译：脉冲神经网络因其在与传统人工神经网络相当准确率的同时，在现实任务中具有低能耗需求而日益受到关注。SNN训练算法面临梯度信息丢失和不可微问题，这是因为在最小化模型参数上的损失时使用了Heaviside函数。为解决这一问题，代理方法在前向传播中使用Heaviside作为脉冲函数，而在反向传播中则采用其可微近似。我们提出在神经元层面使用零阶技术来化解这一二分法，并将其集成到自动微分工具中。由此，我们在所提出的局部零阶技术与现有代理方法之间建立了理论联系，反之亦然。该方法自然适用于在GPU上进行SNN的节能训练。在神经形态数据集上的实验结果表明，该实现中仅需不到1%的神经元在反向传播中保持活跃，从而将反向计算速度提升100倍。与最先进的节能技术相比，我们的方法在保持相似效率的同时具有更好的泛化性能。