The rapid growth of artificial intelligence (AI) has brought novel data processing and generative capabilities but also escalating energy requirements. This challenge motivates renewed interest in neuromorphic computing principles, which promise brain-like efficiency through discrete and sparse activations, recurrent dynamics, and non-linear feedback. In fact, modern AI architectures increasingly embody neuromorphic principles through heavily quantized activations, state-space dynamics, and sparse attention mechanisms. This paper elaborates on the connections between neuromorphic models, state-space models, and transformer architectures through the lens of the distinction between intra-token processing and inter-token processing. Most early work on neuromorphic AI was based on spiking neural networks (SNNs) for intra-token processing, i.e., for transformations involving multiple channels, or features, of the same vector input, such as the pixels of an image. In contrast, more recent research has explored how neuromorphic principles can be leveraged to design efficient inter-token processing methods, which selectively combine different information elements depending on their contextual relevance. Implementing associative memorization mechanisms, these approaches leverage state-space dynamics or sparse self-attention. Along with a systematic presentation of modern neuromorphic AI models through the lens of intra-token and inter-token processing, training methodologies for neuromorphic AI models are also reviewed. These range from surrogate gradients leveraging parallel convolutional processing to local learning rules based on reinforcement learning mechanisms.

翻译：人工智能的快速发展带来了新颖的数据处理和生成能力，但也伴随着不断攀升的能耗需求。这一挑战重新激发了人们对神经形态计算原理的兴趣，该原理通过离散稀疏的激活、循环动力学和非线性反馈，有望实现类脑的高效计算。事实上，现代人工智能架构正通过重度量化的激活、状态空间动力学和稀疏注意力机制，日益体现神经形态原理。本文通过区分令牌内处理与令牌间处理的视角，详细阐述了神经形态模型、状态空间模型与Transformer架构之间的联系。早期关于神经形态人工智能的研究大多基于脉冲神经网络（SNNs）进行令牌内处理，即对同一向量输入（如图像像素）的多个通道或特征进行变换。相比之下，近期的研究探索了如何利用神经形态原理设计高效的令牌间处理方法，这些方法根据信息元素在上下文中的相关性进行选择性组合。通过实现联想记忆机制，这些方法利用了状态空间动力学或稀疏自注意力。除了从令牌内与令牌间处理的角度系统阐述现代神经形态人工智能模型外，本文还综述了神经形态人工智能模型的训练方法，涵盖从利用并行卷积处理的代理梯度到基于强化学习机制的局部学习规则。