The rapid growth of artificial intelligence (AI) has brought novel data processing and generative capabilities but also escalating energy requirements. This challenge motivates renewed interest in neuromorphic computing principles, which promise brain-like efficiency through discrete and sparse activations, recurrent dynamics, and non-linear feedback. In fact, modern AI architectures increasingly embody neuromorphic principles through heavily quantized activations, state-space dynamics, and sparse attention mechanisms. This paper elaborates on the connections between neuromorphic models, state-space models, and transformer architectures through the lens of the distinction between intra-token processing and inter-token processing. Most early work on neuromorphic AI was based on spiking neural networks (SNNs) for intra-token processing, i.e., for transformations involving multiple channels, or features, of the same vector input, such as the pixels of an image. In contrast, more recent research has explored how neuromorphic principles can be leveraged to design efficient inter-token processing methods, which selectively combine different information elements depending on their contextual relevance. Implementing associative memorization mechanisms, these approaches leverage state-space dynamics or sparse self-attention. Along with a systematic presentation of modern neuromorphic AI models through the lens of intra-token and inter-token processing, training methodologies for neuromorphic AI models are also reviewed. These range from surrogate gradients leveraging parallel convolutional processing to local learning rules based on reinforcement learning mechanisms.

翻译：人工智能（AI）的快速发展带来了新颖的数据处理和生成能力，但也伴随着日益增长的能源需求。这一挑战促使人们重新关注神经形态计算原理，该原理通过离散稀疏激活、循环动力学和非线性反馈机制，有望实现类脑的高效计算。事实上，现代AI架构正通过高度量化的激活函数、状态空间动力学和稀疏注意力机制，日益体现出神经形态原理。本文通过区分令牌内处理与令牌间处理的视角，详细阐述了神经形态模型、状态空间模型与Transformer架构之间的关联。早期神经形态AI研究大多基于脉冲神经网络（SNNs）进行令牌内处理，即对同一向量输入（如图像像素）的多通道或多特征进行变换。相比之下，近期研究则探索如何利用神经形态原理设计高效的令牌间处理方法，这些方法能根据上下文相关性选择性地组合不同信息元素。通过实现关联记忆机制，这些方法利用了状态空间动力学或稀疏自注意力机制。本文在系统阐述基于令牌内与令牌间处理的现代神经形态AI模型的同时，也综述了相关训练方法，涵盖从利用并行卷积处理的代理梯度法到基于强化学习机制的局部学习规则。