The rapid growth of artificial intelligence (AI) has brought novel data processing and generative capabilities but also escalating energy requirements. This challenge motivates renewed interest in neuromorphic computing principles, which promise brain-like efficiency through discrete and sparse activations, recurrent dynamics, and non-linear feedback. In fact, modern AI architectures increasingly embody neuromorphic principles through heavily quantized activations, state-space dynamics, and sparse attention mechanisms. This paper elaborates on the connections between neuromorphic models, state-space models, and transformer architectures through the lens of the distinction between intra-token processing and inter-token processing. Most early work on neuromorphic AI was based on spiking neural networks (SNNs) for intra-token processing, i.e., for transformations involving multiple channels, or features, of the same vector input, such as the pixels of an image. In contrast, more recent research has explored how neuromorphic principles can be leveraged to design efficient inter-token processing methods, which selectively combine different information elements depending on their contextual relevance. Implementing associative memorization mechanisms, these approaches leverage state-space dynamics or sparse self-attention. Along with a systematic presentation of modern neuromorphic AI models through the lens of intra-token and inter-token processing, training methodologies for neuromorphic AI models are also reviewed. These range from surrogate gradients leveraging parallel convolutional processing to local learning rules based on reinforcement learning mechanisms.

翻译：人工智能（AI）的快速发展带来了新颖的数据处理和生成能力，但也伴随着不断攀升的能耗需求。这一挑战重新激发了人们对神经形态计算原理的兴趣，该原理通过离散稀疏的激活、循环动力学和非线性反馈，有望实现类脑的高效计算。事实上，现代AI架构正越来越多地通过重度量化的激活、状态空间动力学和稀疏注意力机制来体现神经形态原理。本文通过区分令牌内处理与令牌间处理，详细阐述了神经形态模型、状态空间模型与Transformer架构之间的联系。早期关于神经形态AI的研究大多基于用于令牌内处理的脉冲神经网络（SNNs），即对同一向量输入（如图像像素）的多个通道或特征进行变换。相比之下，近期的研究探索了如何利用神经形态原理来设计高效的令牌间处理方法，这些方法根据上下文相关性选择性地组合不同的信息元素。通过实现联想记忆机制，这些方法利用了状态空间动力学或稀疏自注意力。除了通过令牌内与令牌间处理的视角系统性地介绍现代神经形态AI模型外，本文还回顾了神经形态AI模型的训练方法。这些方法涵盖了从利用并行卷积处理的代理梯度，到基于强化学习机制的局部学习规则。