Spiking neural networks (SNNs), which mimic biological neural system to convey information via discrete spikes, are well known as brain-inspired models with excellent computing efficiency. By utilizing the surrogate gradient estimation for discrete spikes, learning-based SNN training methods that can achieve ultra-low inference latency (number of time-step) emerge recently. Nevertheless, due to the difficulty in deriving precise gradient estimation for discrete spikes using learning-based method, a distinct accuracy gap persists between SNN and its artificial neural networks (ANNs) counterpart. To address the aforementioned issue, we propose a blurred knowledge distillation (BKD) technique, which leverages random blurred SNN feature to restore and imitate the ANN feature. Note that, our BKD is applied upon the feature map right before the last layer of SNN, which can also mix with prior logits-based knowledge distillation for maximized accuracy boost. To our best knowledge, in the category of learning-based methods, our work achieves state-of-the-art performance for training SNNs on both static and neuromorphic datasets. On ImageNet dataset, BKDSNN outperforms prior best results by 4.51% and 0.93% with the network topology of CNN and Transformer respectively.

翻译：脉冲神经网络通过离散脉冲传递信息以模拟生物神经系统，是公认的具有优异计算效率的类脑模型。通过利用离散脉冲的代理梯度估计，近期出现了能够实现超低推理延迟（时间步数）的基于学习的SNN训练方法。然而，由于基于学习的方法难以对离散脉冲推导出精确的梯度估计，SNN与其对应的人工神经网络之间仍存在显著的精度差距。为解决上述问题，我们提出一种模糊知识蒸馏技术，利用随机模糊的SNN特征来恢复并模仿ANN特征。值得注意的是，我们的BKD应用于SNN最后一层之前的特征图，亦可与先验的基于逻辑输出的知识蒸馏相结合以实现最大化的精度提升。据我们所知，在基于学习的方法范畴内，本工作在静态数据集与神经形态数据集上训练SNN均取得了最先进的性能。在ImageNet数据集上，采用CNN与Transformer网络拓扑时，BKDSNN分别以4.51%和0.93%的优势超越了先前最佳结果。