Spiking neural networks (SNN) are able to learn spatiotemporal features while using less energy, especially on neuromorphic hardware. The most widely used spiking neuron in deep learning is the Leaky Integrate and Fire (LIF) neuron. LIF neurons operate sequentially, however, since the computation of state at time t relies on the state at time t-1 being computed. This limitation is shared with Recurrent Neural Networks (RNN) and results in slow training on Graphics Processing Units (GPU). In this paper, we propose the Stochastic Parallelizable Spiking Neuron (SPSN) to overcome the sequential training limitation of LIF neurons. By separating the linear integration component from the non-linear spiking function, SPSN can be run in parallel over time. The proposed approach results in performance comparable with the state-of-the-art for feedforward neural networks on the Spiking Heidelberg Digits (SHD) dataset, outperforming LIF networks while training 10 times faster and outperforming non-spiking networks with the same network architecture. For longer input sequences of 10000 time-steps, we show that the proposed approach results in 4000 times faster training, thus demonstrating the potential of the proposed approach to accelerate SNN training for very large datasets.

翻译：脉冲神经网络（SNN）能够以较低能耗学习时空特征，尤其在神经形态硬件上表现突出。深度学习中最常用的脉冲神经元是泄露积分点火（LIF）神经元。然而，LIF神经元需顺序运行，因为t时刻的状态计算依赖于t-1时刻的状态计算结果。这一限制与循环神经网络（RNN）相同，导致在图形处理器（GPU）上训练速度缓慢。本文提出随机可并行化脉冲神经元（SPSN），以克服LIF神经元的顺序训练限制。通过将线性积分组件与非线性脉冲函数分离，SPSN可实现时间维度上的并行运行。所提方法在Spiking Heidelberg Digits（SHD）数据集上获得与前馈神经网络当前最佳性能相当的结果，训练速度较LIF网络快10倍且性能更优，同时优于采用相同网络架构的非脉冲网络。针对10000时间步的长输入序列，该方法实现4000倍的训练加速，从而展现出加速超大规模数据集SNN训练的潜力。