Spiking neural networks (SNNs), which mimic biological neural system to convey information via discrete spikes, are well known as brain-inspired models with excellent computing efficiency. By utilizing the surrogate gradient estimation for discrete spikes, learning-based SNN training methods that can achieve ultra-low inference latency (number of time-step) emerge recently. Nevertheless, due to the difficulty in deriving precise gradient estimation for discrete spikes using learning-based method, a distinct accuracy gap persists between SNN and its artificial neural networks (ANNs) counterpart. To address the aforementioned issue, we propose a blurred knowledge distillation (BKD) technique, which leverages random blurred SNN feature to restore and imitate the ANN feature. Note that, our BKD is applied upon the feature map right before the last layer of SNN, which can also mix with prior logits-based knowledge distillation for maximized accuracy boost. To our best knowledge, in the category of learning-based methods, our work achieves state-of-the-art performance for training SNNs on both static and neuromorphic datasets. On ImageNet dataset, BKDSNN outperforms prior best results by 4.51% and 0.93% with the network topology of CNN and Transformer respectively.

翻译：脉冲神经网络通过离散脉冲传递信息以模拟生物神经系统，是公认的具有优异计算效率的类脑模型。通过利用离散脉冲的代理梯度估计，近期出现了能够实现超低推理延迟（时间步数）的基于学习的SNN训练方法。然而，由于基于学习的方法难以对离散脉冲推导精确的梯度估计，SNN与其对应的人工神经网络之间仍存在显著的精度差距。为解决上述问题，我们提出了一种模糊知识蒸馏技术，该技术利用随机模糊的SNN特征来恢复并模仿ANN特征。值得注意的是，我们的BKD应用于SNN最后一层之前的特征图，也可与先验的基于逻辑的知识蒸馏相结合以实现最大化的精度提升。据我们所知，在基于学习的方法类别中，我们的工作在使用静态数据集与神经形态数据集训练SNN方面均取得了最先进的性能。在ImageNet数据集上，采用CNN与Transformer网络拓扑的BKDSNN分别以4.51%和0.93%的优势超越了先前的最佳结果。