Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANN): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate Backpropagation using spike-time dependent plasticity (STDP) with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based Back-Prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. We show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a Winner Take All (WTA) rule. Our SNN with a 2-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.

翻译：脉冲神经网络（SNN）已成为分类任务中一种硬件高效的架构。基于尖峰编码的挑战在于缺乏完全使用尖峰实现的通用训练机制。已有多种尝试将非脉冲人工神经网络（ANN）中强大的反向传播（BP）技术应用于此：(1) SNN可通过外部计算数值梯度进行训练；(2) 使用基于尖峰时间依赖可塑性（STDP）的近似反向传播，配合分阶段前向/反向传递，是实现原生尖峰学习的重要进展。然而，此类阶段间用于梯度与权重更新的信息传输需要外部存储与计算访问，这对标准神经形态硬件实现构成挑战。本文提出一种基于随机SNN的反向传播（SSNN-BP）算法，利用复合神经元用尖峰同时显式计算前向传播激活值与反向传播梯度。尽管带符号梯度值对基于尖峰的表示构成挑战，我们通过将梯度信号拆分为正负两路流来解决这一问题。研究表明，该方法在尖峰序列足够长时可逼近BP ANN基线性能。最终证明，性能优异的softmax交叉熵损失函数可通过抑制性侧向连接实现，强制执行胜者全得（WTA）规则。采用2层网络的SNN在MNIST、Fashion-MNIST、扩展MNIST等静态图像数据集以及神经形态MNIST等时序编码图像数据集上，通过等效架构与正则化参数的ANN可比性能展现出卓越泛化能力。因此，SSNN-BP实现了与纯尖峰神经形态硬件兼容的反向传播。