Spiking Neural Networks (SNNs) have gained significant attention due to the energy-efficient and multiplication-free characteristics. Despite these advantages, deploying large-scale SNNs on edge hardware is challenging due to limited resource availability. Network pruning offers a viable approach to compress the network scale and reduce hardware resource requirements for model deployment. However, existing SNN pruning methods cause high pruning costs and performance loss because they lack efficiency in processing the sparse spike representation of SNNs. In this paper, inspired by the critical brain hypothesis in neuroscience and the high biological plausibility of SNNs, we explore and leverage criticality to facilitate efficient pruning in deep SNNs. We firstly explain criticality in SNNs from the perspective of maximizing feature information entropy. Second, We propose a low-cost metric for assess neuron criticality in feature transmission and design a pruning-regeneration method that incorporates this criticality into the pruning process. Experimental results demonstrate that our method achieves higher performance than the current state-of-the-art (SOTA) method with up to 95.26\% reduction of pruning cost. The criticality-based regeneration process efficiently selects potential structures and facilitates consistent feature representation.
翻译:脉冲神经网络(SNNs)因其高能效和无乘法运算的特性而受到广泛关注。尽管具有这些优势,在边缘硬件上部署大规模SNNs仍面临资源有限的挑战。网络剪枝为压缩网络规模、降低模型部署的硬件资源需求提供了可行途径。然而,现有SNN剪枝方法在处理SNNs的稀疏脉冲表示时效率不足,导致高昂的剪枝成本和性能损失。本文受神经科学中临界大脑假说及SNNs高生物合理性的启发,探索并利用临界性以促进深度SNNs的高效剪枝。我们首先从最大化特征信息熵的角度阐释SNNs中的临界性。其次,提出一种用于评估神经元在特征传递中临界性的低代价度量指标,并设计了一种将临界性融入剪枝过程的剪枝-再生方法。实验结果表明,本方法在将剪枝成本降低高达95.26%的同时,性能优于当前最先进(SOTA)方法。基于临界性的再生过程能有效筛选潜在结构,并促进一致的特征表示。