Spiking Neural Networks (SNNs) use discrete spike sequences to transmit information, which significantly mimics the information transmission of the brain. Although this binarized form of representation dramatically enhances the energy efficiency and robustness of SNNs, it also leaves a large gap between the performance of SNNs and Artificial Neural Networks based on real values. There are many different spike patterns in the brain, and the dynamic synergy of these spike patterns greatly enriches the representation capability. Inspired by spike patterns in biological neurons, this paper introduces the dynamic Burst pattern and designs the Leaky Integrate and Fire or Burst (LIFB) neuron that can make a trade-off between short-time performance and dynamic temporal performance from the perspective of network information capacity. LIFB neuron exhibits three modes, resting, Regular spike, and Burst spike. The burst density of the neuron can be adaptively adjusted, which significantly enriches the characterization capability. We also propose a decoupling method that can losslessly decouple LIFB neurons into equivalent LIF neurons, which demonstrates that LIFB neurons can be efficiently implemented on neuromorphic hardware. We conducted experiments on the static datasets CIFAR10, CIFAR100, and ImageNet, which showed that we greatly improved the performance of the SNNs while significantly reducing the network latency. We also conducted experiments on neuromorphic datasets DVS-CIFAR10 and NCALTECH101 and showed that we achieved state-of-the-art with a small network structure.
翻译:脉冲神经网络(SNNs)使用离散脉冲序列传输信息,这极大地模仿了大脑的信息传递方式。尽管这种二值化表征形式显著提升了SNNs的能量效率和鲁棒性,但也导致SNNs与基于实值的人工神经网络之间存在巨大性能差距。大脑中存在多种不同的脉冲模式,这些脉冲模式的动态协同极大地丰富了表征能力。受生物神经元脉冲模式的启发,本文引入了动态爆发模式,并从网络信息容量的角度设计了泄漏积分点火或爆发(LIFB)神经元,该神经元能够在短期性能与动态时序性能之间进行权衡。LIFB神经元表现出三种模式:静息、规则脉冲和爆发脉冲。神经元的爆发密度可以自适应调整,显著增强了表征能力。我们还提出了一种解耦方法,可以将LIFB神经元无损地解耦为等效的LIF神经元,这表明LIFB神经元可以在神经形态硬件上高效实现。我们在静态数据集CIFAR10、CIFAR100和ImageNet上进行了实验,结果显示我们在显著降低网络延迟的同时大幅提升了SNNs的性能。我们还在神经形态数据集DVS-CIFAR10和NCALTECH101上进行了实验,表明我们在小网络结构下达到了最优性能。