Communication by rare, binary spikes is a key factor for the energy efficiency of biological brains. However, it is harder to train biologically-inspired spiking neural networks (SNNs) than artificial neural networks (ANNs). This is puzzling given that theoretical results provide exact mapping algorithms from ANNs to SNNs with time-to-first-spike (TTFS) coding. In this paper we analyze in theory and simulation the learning dynamics of TTFS-networks and identify a specific instance of the vanishing-or-exploding gradient problem. While two choices of SNN mappings solve this problem at initialization, only the one with a constant slope of the neuron membrane potential at threshold guarantees the equivalence of the training trajectory between SNNs and ANNs with rectified linear units. We demonstrate that training deep SNN models achieves the exact same performance as that of ANNs, surpassing previous SNNs on image classification datasets such as MNIST/Fashion-MNIST, CIFAR10/CIFAR100 and PLACES365. Our SNN accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation. We show that fine-tuning SNNs with our robust gradient descent algorithm enables their optimization for hardware implementations with low latency and resilience to noise and quantization.

翻译：通过罕见、二值脉冲进行通信是生物大脑能量高效的关键因素。然而，相较于人工神经网络（ANNs），受生物启发的脉冲神经网络（SNNs）的训练难度更大。考虑到理论结果已提供从ANNs到采用首次脉冲时间（TTFS）编码的SNNs的精确映射算法，这一现象令人费解。本文从理论和仿真角度分析TTFS网络的动力学特性，并指出梯度消失或爆炸问题的一个特定实例。尽管两种SNN映射选择可在初始化阶段解决该问题，但仅当神经元膜电位在阈值处具有恒定斜率时，才能确保SNN与采用整流线性单元的ANN在训练轨迹上的等价性。我们证明，深度SNN模型的训练性能与ANN完全一致，在MNIST/Fashion-MNIST、CIFAR10/CIFAR100和PLACES365等图像分类数据集上超越此前所有SNN方法。我们的SNN以每个神经元少于0.3个脉冲实现高性能分类，适用于高能效硬件实现。研究表明，通过所提出的鲁棒梯度下降算法微调SNN，可使其针对低延迟、抗噪声与抗量化的硬件实现进行优化。