Spiking Neural Networks (SNNs) have gained considerable attention due to the energy-efficient and multiplication-free characteristics. The continuous growth in scale of deep SNNs poses challenges for model deployment. Network pruning reduces hardware resource requirements of model deployment by compressing the network scale. However, existing SNN pruning methods cause high pruning costs and performance loss because the pruning iterations amplify the training difficulty of SNNs. In this paper, inspired by the critical brain hypothesis in neuroscience, we propose a regeneration mechanism based on the neuron criticality for SNN pruning to enhance feature extraction and accelerate the pruning process. Firstly, we propose a low-cost metric for the criticality in SNNs. Then, we re-rank the pruned structures after pruning and regenerate those with higher criticality to obtain the critical network. Our method achieves higher performance than the current state-of-the-art (SOTA) method with up to 95.26% reduction of pruning cost. Moreover, we investigate the underlying mechanism of our method and find that it efficiently selects potential structures and learns the consistent feature representation.
翻译:脉冲神经网络(SNNs)凭借其能效高且无需乘法的特性,已获得广泛关注。然而,深度SNN规模的持续增长为模型部署带来了挑战。网络剪枝通过压缩网络规模降低了模型部署的硬件资源需求。但现有SNN剪枝方法因剪枝迭代放大了SNN的训练难度,导致剪枝成本高昂及性能损失。受神经科学中大脑临界假说的启发,本文提出一种基于神经元临界性的再生机制用于SNN剪枝,以增强特征提取并加速剪枝过程。首先,我们提出了一种低成本的SNN临界性度量方法;其次,在剪枝后对已剪枝结构进行重新排序,并再生临界性较高的结构以获得临界网络。该方法在剪枝成本最高降低95.26%的情况下,实现了优于当前最优方法(SOTA)的性能。此外,我们探讨了该方法的内在机理,发现其能高效筛选潜在结构并学习一致的特征表示。