The human brain exhibits remarkable abilities in integrating temporally distant sensory inputs for decision-making. However, existing brain-inspired spiking neural networks (SNNs) have struggled to match their biological counterpart in modeling long-term temporal relationships. To address this problem, this paper presents a novel Contextual Embedding Leaky Integrate-and-Fire (CE-LIF) spiking neuron model. Specifically, the CE-LIF model incorporates a meticulously designed contextual embedding component into the adaptive neuronal firing threshold, thereby enhancing the memory storage of spiking neurons and facilitating effective sequential modeling. Additionally, theoretical analysis is provided to elucidate how the CE-LIF model enables long-term temporal credit assignment. Remarkably, when compared to state-of-the-art recurrent SNNs, feedforward SNNs comprising the proposed CE-LIF neurons demonstrate superior performance across extensive sequential modeling tasks in terms of classification accuracy, network convergence speed, and memory capacity.

翻译：人脑在整合时间上遥远的感官输入以进行决策方面表现出卓越的能力。然而，现有受脑启发的脉冲神经网络在建模长时程时间关系方面难以与其生物对应物相匹配。为解决这一问题，本文提出了一种新颖的上下文嵌入泄露整合发放（CE-LIF）脉冲神经元模型。具体而言，CE-LIF模型将精心设计的上下文嵌入组件融入自适应神经元发放阈值中，从而增强脉冲神经元的记忆存储能力并促进有效的序列建模。此外，本文提供理论分析以阐明CE-LIF模型如何实现长时程时间信用分配。值得注意的是，与最先进的递归型SNN相比，包含所提出CE-LIF神经元的前馈型SNN在分类精度、网络收敛速度和记忆容量等广泛的序列建模任务中展现出更优越的性能。