Spiking Neural Networks (SNNs) offer a promising avenue for energy-efficient computing compared with Artificial Neural Networks (ANNs), closely mirroring biological neural processes. However, this potential comes with inherent challenges in directly training SNNs through spatio-temporal backpropagation -- stemming from the temporal dynamics of spiking neurons and their discrete signal processing -- which necessitates alternative ways of training, most notably through ANN-SNN conversion. In this work, we introduce a lightweight Forward Temporal Bias Correction (FTBC) technique, aimed at enhancing conversion accuracy without the computational overhead. We ground our method on provided theoretical findings that through proper temporal bias calibration the expected error of ANN-SNN conversion can be reduced to be zero after each time step. We further propose a heuristic algorithm for finding the temporal bias only in the forward pass, thus eliminating the computational burden of backpropagation and we evaluate our method on CIFAR-10/100 and ImageNet datasets, achieving a notable increase in accuracy on all datasets. Codes are released at a GitHub repository.
翻译:脉冲神经网络(SNN)与人工神经网络(ANN)相比,为节能计算提供了有前景的途径,其运作机制紧密模拟生物神经过程。然而,这一潜力伴随直接通过时空反向传播训练SNN的固有挑战——源于脉冲神经元的时序动态特性及其离散信号处理方式——这迫使需要替代训练方法,其中最典型的是通过ANN-SNN转换。本文提出一种轻量级的前向时间偏置校正(FTBC)技术,旨在无需计算开销的前提下提升转换精度。我们的方法基于提供的理论发现:通过适当的时间偏置校准,ANN-SNN转换的预期误差可在每个时间步后降至零。我们进一步提出一种启发式算法,仅在前向传播中寻找时间偏置,从而消除反向传播的计算负担,并在CIFAR-10/100和ImageNet数据集上评估方法,所有数据集的精度均显著提升。代码已发布于GitHub仓库。