Spiking neural networks (SNNs) are artificial neural networks based on simulated biological neurons and have attracted much attention in recent artificial intelligence technology studies. The dendrites in biological neurons have efficient information processing ability and computational power; however, the neurons of SNNs rarely match the complex structure of the dendrites. Inspired by the nonlinear structure and highly sparse properties of neuronal dendrites, in this study, we propose an efficient, lightweight SNN method with nonlinear pruning and dendritic integration (NSPDI-SNN). In this method, we introduce nonlinear dendritic integration (NDI) to improve the representation of the spatiotemporal information of neurons. We implement heterogeneous state transition ratios of dendritic spines and construct a new and flexible nonlinear synaptic pruning (NSP) method to achieve the high sparsity of SNN. We conducted systematic experiments on three benchmark datasets (DVS128 Gesture, CIFAR10-DVS, and CIFAR10) and extended the evaluation to two complex tasks (speech recognition and reinforcement learning-based maze navigation task). Across all tasks, NSPDI-SNN consistently achieved high sparsity with minimal performance degradation. In particular, our method achieved the best experimental results on all three event stream datasets. Further analysis showed that NSPDI significantly improved the efficiency of synaptic information transfer as sparsity increased. In conclusion, our results indicate that the complex structure and nonlinear computation of neuronal dendrites provide a promising approach for developing efficient SNN methods.
翻译:脉冲神经网络(SNN)是基于模拟生物神经元的人工神经网络,近年来在人工智能技术研究中备受关注。生物神经元中的树突具有高效的信息处理能力和计算能力;然而,SNN的神经元很少匹配树突的复杂结构。受神经元树突的非线性结构和高度稀疏特性的启发,本研究提出了一种基于非线性剪枝与树突整合的高效轻量化SNN方法(NSPDI-SNN)。在该方法中,我们引入非线性树突整合(NDI)以提升神经元时空信息的表征能力。我们实现了树突棘的异质性状态转移比,并构建了一种新颖灵活的非线性突触剪枝(NSP)方法,以实现SNN的高稀疏性。我们在三个基准数据集(DVS128 Gesture、CIFAR10-DVS和CIFAR10)上进行了系统实验,并将评估扩展到两项复杂任务(语音识别和基于强化学习的迷宫导航任务)。在所有任务中,NSPDI-SNN均能以极小的性能损失持续实现高稀疏性。特别地,我们的方法在所有三个事件流数据集上均取得了最佳实验结果。进一步分析表明,随着稀疏度的增加,NSPDI显著提升了突触信息传递的效率。综上所述,我们的研究结果表明,神经元树突的复杂结构和非线性计算为开发高效的SNN方法提供了一条有前景的途径。