Spiking Neural Network (SNN) as a brain-inspired strategy receives lots of attention because of the high-sparsity and low-power properties derived from its inherent spiking information state. To further improve the efficiency of SNN, some works declare that the Lottery Tickets (LTs) Hypothesis, which indicates that the Artificial Neural Network (ANN) contains a subnetwork without sacrificing the performance of the original network, also exists in SNN. However, the spiking information handled by SNN has a natural similarity and affinity with binarization in sparsification. Therefore, to further explore SNN efficiency, this paper focuses on (1) the presence or absence of LTs in the binary SNN, and (2) whether the spiking mechanism is a superior strategy in terms of handling binary information compared to simple model binarization. To certify these consumptions, a sparse training method is proposed to find Binary Weights Spiking Lottery Tickets (BinW-SLT) under different network structures. Through comprehensive evaluations, we show that BinW-SLT could attain up to +5.86% and +3.17% improvement on CIFAR-10 and CIFAR-100 compared with binary LTs, as well as achieve 1.86x and 8.92x energy saving compared with full-precision SNN and ANN.
翻译:脉冲神经网络(SNN)作为一种受大脑启发的策略,由于其固有的脉冲信息状态具有高稀疏性和低功耗特性而备受关注。为了进一步提升SNN的效率,一些研究指出,人工神经网络(ANN)中存在一种在不损失原网络性能的前提下能够提取子网络的“中奖票”(LTs)假设,这种假设同样适用于SNN。然而,SNN处理的脉冲信息在稀疏化过程中与二值化具有天然的相似性和亲和性。因此,为了进一步探索SNN的效率,本文聚焦于:(1)二元SNN中是否存在LTs,以及(2)与简单模型二值化相比,脉冲机制在处理二元信息方面是否是一种更优策略。为了验证这些假设,提出了一种稀疏训练方法,用于在不同网络结构下寻找二元权值脉冲中奖票(BinW-SLT)。通过全面评估,我们表明,与二元LTs相比,BinW-SLT在CIFAR-10和CIFAR-100数据集上分别提升了高达+5.86%和+3.17%的性能,同时与全精度SNN和ANN相比,实现了1.86倍和8.92倍的能耗节省。