Recurrent spiking neural networks (RSNNs) are notoriously difficult to train because of the vanishing gradient problem that is enhanced by the binary nature of the spikes. In this paper, we review the ability of the current state-of-the-art RSNNs to solve long-term memory tasks, and show that they have strong constraints both in performance, and for their implementation on hardware analog neuromorphic processors. We present a novel spiking neural network that circumvents these limitations. Our biologically inspired neural network uses synaptic delays, branching factor regularization and a novel surrogate derivative for the spiking function. The proposed network proves to be more successful in using the recurrent connections on memory tasks.

翻译：循环脉冲神经网络因尖峰的二元性质而加剧的梯度消失问题，使其训练难度极高。本文回顾了当前最先进的循环脉冲神经网络在解决长期记忆任务中的能力，并表明其在性能及硬件模拟神经形态处理器实现方面均存在显著限制。我们提出一种新型脉冲神经网络，可规避这些局限。该生物启发式神经网络采用了突触延迟、分支因子正则化及一种新型尖峰函数代理导数。实验证明，所提网络在记忆任务中能更有效地利用循环连接。