The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its "Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the "Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the "Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.
翻译:脉冲神经网络(SNN)近年来受到越来越多的关注。它采用二进制脉冲信号来传递信息。得益于SNN的信息传递范式,激活值与权重的乘法运算可以替换为加法运算,从而更加节能。然而,其发放活动的“硬重置”机制会忽略膜电位高于发放阈值时的差异,导致信息损失。同时,在发放时刻将膜电位量化为0/1脉冲不可避免地引入量化误差,同样造成信息损失。为解决这些问题,我们提出对基于监督训练的SNN采用“软重置”机制,该机制会根据膜电位幅值将其驱动至动态重置电位,并引入膜电位整流器(MPR),通过将膜电位重新分布至接近脉冲的范围内来减少量化误差。结果表明,采用“软重置”机制和MPR的SNN在静态与动态数据集上均优于其原始版本。