Brain-inspired Spiking Neural Networks (SNNs) have the characteristics of event-driven and high energy-efficient, which are different from traditional Artificial Neural Networks (ANNs) when deployed on edge devices such as neuromorphic chips. Most previous work focuses on SNNs training strategies to improve model performance and brings larger and deeper network architectures. It is difficult to deploy these complex networks on resource-limited edge devices directly. To meet such demand, people compress SNNs very cautiously to balance the performance and the computation efficiency. Existing compression methods either iteratively pruned SNNs using weights norm magnitude or formulated the problem as a sparse learning optimization. We propose an improved end-to-end Minimax optimization method for this sparse learning problem to better balance the model performance and the computation efficiency. We also demonstrate that jointly applying compression and finetuning on SNNs is better than sequentially, especially for extreme compression ratios. The compressed SNN models achieved state-of-the-art (SOTA) performance on various benchmark datasets and architectures. Our code is available at https://github.com/chenjallen/Resource-Constrained-Compression-on-SNN.

翻译：受大脑启发的脉冲神经网络（SNNs）具有事件驱动和高能效特性，这与传统人工神经网络（ANNs）在神经形态芯片等边缘设备上部署时有所不同。以往研究大多聚焦于SNNs的训练策略以提升模型性能，导致网络架构更庞大、更深。这些复杂网络难以直接部署在资源受限的边缘设备上。为满足这一需求，人们谨慎地压缩SNNs以平衡性能与计算效率。现有的压缩方法要么基于权值范数幅度迭代剪枝，要么将问题表述为稀疏学习优化。我们针对该稀疏学习问题提出一种改进的端到端极小极大优化方法，以更好地平衡模型性能与计算效率。实验结果还表明，在SNNs上联合应用压缩与微调优于顺序进行，尤其在极端压缩比下表现更为显著。所压缩的SNN模型在多个基准数据集和架构上均实现了最先进的性能。我们的代码开源地址为：https://github.com/chenjallen/Resource-Constrained-Compression-on-SNN。