Redundant information transfer in a neural network can increase the complexity of the deep learning model, thus increasing its power consumption. We introduce in this paper a novel spiking neuron, termed Variable Spiking Neuron (VSN), which can reduce the redundant firing using lessons from biological neuron inspired Leaky Integrate and Fire Spiking Neurons (LIF-SN). The proposed VSN blends LIF-SN and artificial neurons. It garners the advantage of intermittent firing from the LIF-SN and utilizes the advantage of continuous activation from the artificial neuron. This property of the proposed VSN makes it suitable for regression tasks, which is a weak point for the vanilla spiking neurons, all while keeping the energy budget low. The proposed VSN is tested against both classification and regression tasks. The results produced advocate favorably towards the efficacy of the proposed spiking neuron, particularly for regression tasks.
翻译:神经网络中的信息冗余传输会增加深度学习模型的复杂度,从而提升其功耗。本文提出一种新型脉冲神经元——可变脉冲神经元(Variable Spiking Neuron, VSN),该神经元借鉴生物神经元启发的泄露积分触发脉冲神经元(Leaky Integrate and Fire Spiking Neuron, LIF-SN)原理,能够减少冗余脉冲发放。所提出的VSN融合了LIF-SN与人工神经元的特性:既继承了LIF-SN间歇性脉冲发放的优势,又利用了人工神经元连续激活的特点。这种特性使VSN在保持低能耗的同时,特别适用于回归任务——而这正是传统脉冲神经元的薄弱环节。我们在分类与回归两种任务上对VSN进行测试,实验结果充分证明了所提出脉冲神经元的有效性,尤其在回归任务中表现优异。