The event-driven and sparse nature of communication between spiking neurons in the brain holds great promise for flexible and energy-efficient AI. Recent advances in learning algorithms have demonstrated that recurrent networks of spiking neurons can be effectively trained to achieve competitive performance compared to standard recurrent neural networks. Still, as these learning algorithms use error-backpropagation through time (BPTT), they suffer from high memory requirements, are slow to train, and are incompatible with online learning. This limits the application of these learning algorithms to relatively small networks and to limited temporal sequence lengths. Online approximations to BPTT with lower computational and memory complexity have been proposed (e-prop, OSTL), but in practice also suffer from memory limitations and, as approximations, do not outperform standard BPTT training. Here, we show how a recently developed alternative to BPTT, Forward Propagation Through Time (FPTT) can be applied in spiking neural networks. Different from BPTT, FPTT attempts to minimize an ongoing dynamically regularized risk on the loss. As a result, FPTT can be computed in an online fashion and has fixed complexity with respect to the sequence length. When combined with a novel dynamic spiking neuron model, the Liquid-Time-Constant neuron, we show that SNNs trained with FPTT outperform online BPTT approximations, and approach or exceed offline BPTT accuracy on temporal classification tasks. This approach thus makes it feasible to train SNNs in a memory-friendly online fashion on long sequences and scale up SNNs to novel and complex neural architectures.

翻译：脑中弹跳神经元之间的事件驱动和稀疏的通信性质为灵活和节能的AI带来了巨大的希望。最近学习算法的进步表明,与标准的经常性神经网络相比,反复涌动神经元的网络可以进行有效的培训,以达到与标准的经常性神经网络相比的竞争性性能。然而,由于这些学习算法通过时间错误背反演化(BPTTT),它们受到高记忆要求的困扰,培训速度缓慢,与在线学习不相容。这限制了这些学习算法的应用,使之局限于相对较小的网络和有限的时间序列长度。对计算准确性和记忆复杂性较低的BPTTT的在线近似(e-op,OSTL)已经提出,但实际上,由于内存限制和近似性能,这些神经网络网络网络网络网络网络化网络化网络化网络化网络化(PTTT)的最近开发的替代方法,在培训的S-NTTF时,S-ral-ral-ral-ral-ral-ral-ral-ral-ral-ral-ral-ral-ral-slick-slick-slick-slation-xxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,在S-xxxxxxxxxxx,在S-x,在