As the scales of neural networks increase, techniques that enable them to run with low computational cost and energy efficiency are required. From such demands, various efficient neural network paradigms, such as spiking neural networks (SNNs) or binary neural networks (BNNs), have been proposed. However, they have sticky drawbacks, such as degraded inference accuracy and latency. To solve these problems, we propose a single-step spiking neural network (S$^3$NN), an energy-efficient neural network with low computational cost and high precision. The proposed S$^3$NN processes the information between hidden layers by spikes as SNNs. Nevertheless, it has no temporal dimension so that there is no latency within training and inference phases as BNNs. Thus, the proposed S$^3$NN has a lower computational cost than SNNs that require time-series processing. However, S$^3$NN cannot adopt na\"{i}ve backpropagation algorithms due to the non-differentiability nature of spikes. We deduce a suitable neuron model by reducing the surrogate gradient for multi-time step SNNs to a single-time step. We experimentally demonstrated that the obtained surrogate gradient allows S$^3$NN to be trained appropriately. We also showed that the proposed S$^3$NN could achieve comparable accuracy to full-precision networks while being highly energy-efficient.

翻译：随着神经网络规模增大，需要能够以低计算成本和能效运行的技术。为满足此类需求，研究者提出了多种高效神经网络范式，例如脉冲神经网络（SNN）或二值神经网络（BNN）。然而，这些网络存在推理精度下降和延迟等固有问题。为解决上述问题，我们提出单步脉冲神经网络（S$^3$NN），这是一种兼具低计算成本和高精度的能效神经网络。所提S$^3$NN像SNN一样通过脉冲在隐层间传递信息，但因其不含时间维度，故与BNN相同，在训练和推理阶段均无延迟。因此，S$^3$NN比需要时序处理的SNN具有更低的计算成本。然而，由于脉冲的不可微特性，S$^3$NN无法直接采用朴素反向传播算法。我们将多时间步SNN的替代梯度缩减至单时间步，推导出适配的神经元模型。实验证明，所获得的替代梯度能够使S$^3$NN得到适当训练。我们还表明，所提S$^3$NN在保持高度能效的同时，可达到与全精度网络相当的精度。