Spiking neural networks are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore drastically decrease energy consumption when run on specialised hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art spiking neural networks are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about spiking neural networks focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network to make it event-based. This new RNN cell, called the Spiking Recurrent Cell, therefore communicates using events, i.e. spikes, while being completely differentiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.

翻译：脉冲神经网络是一种人工神经网络，其中神经元之间的通信仅通过事件（也称为脉冲）进行。这一特性使神经网络能够实现异步和稀疏计算，从而在专用硬件上运行时大幅降低能耗。然而，训练此类网络通常较为困难，主要源于脉冲激活函数的不可微性，这阻碍了经典反向传播算法的应用。这是因为当前最先进的脉冲神经网络通常源自生物启发的神经元模型，并在此基础上应用机器学习方法进行训练。目前，关于脉冲神经网络的研究主要聚焦于设计训练算法，其目标是获得在特定任务上能与非脉冲版本相竞争的神经网络。本文尝试了对称方法：我们修改了一种经典且易于训练循环神经网络的动力学特性，使其基于事件进行通信。这种新型循环神经网络单元（被称为脉冲循环单元）能够通过事件（即脉冲）进行通信，同时保持完全可微性。因此，可以使用标准反向传播算法训练由这种循环单元构成的任何网络。实验表明，该新型网络在MNIST基准测试及其变体Fashion-MNIST和Neuromorphic-MNIST上，能够达到与其他类型脉冲网络相当的性能。此外，我们还证明这种新型单元使得深度脉冲网络的训练成为可能。