The Spiking Neural Network (SNN), as one of the biologically inspired neural network infrastructures, has drawn increasing attention recently. It adopts binary spike activations to transmit information, thus the multiplications of activations and weights can be substituted by additions, which brings high energy efficiency. However, in the paper, we theoretically and experimentally prove that the binary spike activation map cannot carry enough information, thus causing information loss and resulting in accuracy decreasing. To handle the problem, we propose a ternary spike neuron to transmit information. The ternary spike neuron can also enjoy the event-driven and multiplication-free operation advantages of the binary spike neuron but will boost the information capacity. Furthermore, we also embed a trainable factor in the ternary spike neuron to learn the suitable spike amplitude, thus our SNN will adopt different spike amplitudes along layers, which can better suit the phenomenon that the membrane potential distributions are different along layers. To retain the efficiency of the vanilla ternary spike, the trainable ternary spike SNN will be converted to a standard one again via a re-parameterization technique in the inference. Extensive experiments with several popular network structures over static and dynamic datasets show that the ternary spike can consistently outperform state-of-the-art methods. Our code is open-sourced at https://github.com/yfguo91/Ternary-Spike.

翻译：尖峰神经网络作为一类受生物启发的神经网络架构，近来受到越来越多的关注。它采用二元尖峰激活来传递信息，因此激活值与权重的乘法可被加法替代，从而带来高能效优势。然而，本文通过理论与实验证明，二元尖峰激活图无法携带足够的信息，导致信息损失和精度下降。为解决这一问题，我们提出一种三元尖峰神经元来传递信息。三元尖峰神经元既能保留二元尖峰神经元的事件驱动与免乘法运算优势，又能提升信息容量。此外，我们在三元尖峰神经元中嵌入可训练因子以学习合适的尖峰幅值，使尖峰神经网络沿不同层采用不同的尖峰幅值，从而更好地适应膜电位分布随层变化的现象。为保持原始三元尖峰的效率，可训练三元尖峰网络将在推理阶段通过重参数化技术重新转换为标准网络。在静态与动态数据集上基于多种主流网络结构的广泛实验表明，三元尖峰方法能持续超越现有最优方法。我们的代码已开源至 https://github.com/yfguo91/Ternary-Spike。