The Spiking Neural Network (SNN), as one of the biologically inspired neural network infrastructures, has drawn increasing attention recently. It adopts binary spike activations to transmit information, thus the multiplications of activations and weights can be substituted by additions, which brings high energy efficiency. However, in the paper, we theoretically and experimentally prove that the binary spike activation map cannot carry enough information, thus causing information loss and resulting in accuracy decreasing. To handle the problem, we propose a ternary spike neuron to transmit information. The ternary spike neuron can also enjoy the event-driven and multiplication-free operation advantages of the binary spike neuron but will boost the information capacity. Furthermore, we also embed a trainable factor in the ternary spike neuron to learn the suitable spike amplitude, thus our SNN will adopt different spike amplitudes along layers, which can better suit the phenomenon that the membrane potential distributions are different along layers. To retain the efficiency of the vanilla ternary spike, the trainable ternary spike SNN will be converted to a standard one again via a re-parameterization technique in the inference. Extensive experiments with several popular network structures over static and dynamic datasets show that the ternary spike can consistently outperform state-of-the-art methods. Our code is open-sourced at https://github.com/yfguo91/Ternary-Spike.

翻译：脉冲神经网络（SNN）作为受生物启发的神经网络基础设施之一，近年来受到越来越多的关注。它采用二元脉冲激活传递信息，因此激活值与权重的乘法运算可被加法替代，从而实现高能效。然而，本文通过理论与实验证明：二元脉冲激活图无法承载足够信息，导致信息损失和精度下降。为解决这一问题，我们提出三元脉冲神经元用于信息传递。该神经元既保留了二元脉冲神经元的事件驱动与免乘法运算优势，又可提升信息容量。此外，我们在三元脉冲神经元中嵌入可训练因子，用于学习合适的脉冲幅值，从而使SNN各层采用不同脉冲幅值，更好地适应膜电位分布沿层变化的特性。为保留原始三元脉冲的运算效率，可训练三元脉冲SNN在推理阶段通过重参数化技术重新转换为标准形式。在静态与动态数据集上采用多种主流网络结构进行的广泛实验表明，三元脉冲始终优于现有最先进方法。我们的代码已开源在https://github.com/yfguo91/Ternary-Spike。