Spiking neural networks are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore drastically decrease energy consumption when run on specialized hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art spiking neural networks are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about spiking neural networks focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network to make it event-based. This new RNN cell, called the Spiking Recurrent Cell, therefore communicates using events, i.e. spikes, while being completely differentiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.

翻译：脉冲神经网络是一种人工神经网络，其中神经元之间的通信仅通过事件（即脉冲）进行。这一特性使得神经网络能够进行异步和稀疏计算，从而在专用硬件上运行时大幅降低能耗。然而，训练此类网络通常较为困难，主要问题在于脉冲激活函数的不可微性阻碍了经典反向传播算法的应用。这是因为当前最先进的脉冲神经网络通常源于生物启发式神经元模型，并在此基础上应用机器学习方法进行训练。目前，脉冲神经网络的研究重点在于设计训练算法，旨在使网络在特定任务上能够与非脉冲版本相竞争。本文尝试从对称角度出发：我们修改了一种经典且易于训练的循环神经网络的动态机制，使其变为基于事件的模型。这种新型循环神经网络单元（称为脉冲循环单元）因而通过事件（即脉冲）进行通信，同时保持完全可微性，因此可以使用标准反向传播算法训练由该单元构成的任意网络。实验表明，在MNIST基准测试及其变体Fashion-MNIST和Neuromorphic-MNIST上，该新型网络能够达到与其他脉冲网络相当的性能。此外，我们还证明该新型单元能够有效实现深度脉冲网络的训练。