Spiking Neural Networks (SNNs) have emerged as a promising third generation of neural networks, offering unique characteristics such as binary outputs, high sparsity, and biological plausibility. However, the lack of effective learning algorithms remains a challenge for SNNs. For instance, while converting artificial neural networks (ANNs) to SNNs circumvents the need for direct training of SNNs, it encounters issues related to conversion errors and high inference time delays. In order to reduce or even eliminate conversion errors while decreasing inference time-steps, we have introduced a novel type of neuron called Group Neurons (GNs). One GN is composed of multiple Integrate-and-Fire (IF) neurons as members, and its neural dynamics are meticulously designed. Based on GNs, we have optimized the traditional ANN-SNN conversion framework. Specifically, we replace the IF neurons in the SNNs obtained by the traditional conversion framework with GNs. The resulting SNNs, which utilize GNs, are capable of achieving accuracy levels comparable to ANNs even within extremely short inference time-steps. The experiments on CIFAR10, CIFAR100, and ImageNet datasets demonstrate the superiority of the proposed methods in terms of both inference accuracy and latency. Code is available at https://github.com/Lyu6PosHao/ANN2SNN_GN.
翻译:脉冲神经网络(SNNs)作为第三代神经网络,具有二值输出、高稀疏性和生物可解释性等独特特性。然而,缺乏有效学习算法仍是SNNs面临的挑战。例如,通过将人工神经网络(ANNs)转换为SNNs虽能规避直接训练SNNs的问题,但会引发转换误差和高推理延迟等缺陷。为减少甚至消除转换误差并降低推理时间步长,我们提出一种新型神经元——群神经元(Group Neurons, GNs)。每个GN由多个整合发放(IF)神经元作为成员组成,并精心设计了其神经动力学机制。基于GNs,我们对传统ANN-SNN转换框架进行了优化。具体而言,我们将传统转换方法获得的SNN中的IF神经元替换为GNs。这种采用GNs的SNN即使在极短推理时间步长内也能达到与ANN相当的精度。在CIFAR10、CIFAR100和ImageNet数据集上的实验证明,所提方法在推理精度和延迟方面均具有优越性。代码开源地址:https://github.com/Lyu6PosHao/ANN2SNN_GN。