Spiking neural networks are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore to drastically decrease energy consumption when run on specialized hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art spiking neural networks are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about spiking neural networks focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network to make it event-based. This new RNN cell, called the Spiking Recurrent Cell, therefore communicates using events, i.e. spikes, while being completely differentiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.

翻译：脉冲神经网络是一类人工神经网络，其神经元间通信仅通过事件（即脉冲）进行。这一特性使得神经网络能够实现异步稀疏计算，从而在专用硬件上大幅降低能耗。然而，训练此类网络具有公认的难度，主要源于脉冲激活函数的不可微性，这阻碍了经典反向传播算法的应用。这是因为当前最先进的脉冲神经网络通常源自生物启发式神经元模型，并采用机器学习方法进行训练。当前关于脉冲神经网络的研究重点在于设计训练算法，旨在使网络在特定任务上达到与非脉冲版本相竞争的性能。本文采用对称性研究思路：对一类经典且易于训练的递归神经网络进行动力学改进，使其实现事件驱动计算。这种新型递归神经网络单元——脉冲递归单元——在保持完全可微性的同时，通过事件（即脉冲）进行通信。因此，任何由该单元构成的网络均可采用标准反向传播进行训练。实验表明，该新型网络在MNIST基准测试及其变体Fashion-MNIST和Neuromorphic-MNIST上可达到与其他脉冲神经网络相当的性能。此外，本研究还证实该单元能够实现深度脉冲网络的有效训练。