Recent years have witnessed a growing call for renewed emphasis on neuroscience-inspired approaches in artificial intelligence research, under the banner of NeuroAI. A prime example of this is predictive coding networks (PCNs), based on the neuroscientific framework of predictive coding. This framework views the brain as a hierarchical Bayesian inference model that minimizes prediction errors through feedback connections. Unlike traditional neural networks trained with backpropagation (BP), PCNs utilize inference learning (IL), a more biologically plausible algorithm that explains patterns of neural activity that BP cannot. Historically, IL has been more computationally intensive, but recent advancements have demonstrated that it can achieve higher efficiency than BP with sufficient parallelization. Furthermore, PCNs can be mathematically considered a superset of traditional feedforward neural networks (FNNs), significantly extending the range of trainable architectures. As inherently probabilistic (graphical) latent variable models, PCNs provide a versatile framework for both supervised learning and unsupervised (generative) modeling that goes beyond traditional artificial neural networks. This work provides a comprehensive review and detailed formal specification of PCNs, particularly situating them within the context of modern ML methods. This positions PC as a promising framework for future ML innovations.

翻译：近年来，在NeuroAI的旗帜下，人工智能研究领域日益呼吁重新重视受神经科学启发的途径。预测编码网络（PCNs）便是其中的典型代表，它基于预测编码的神经科学框架。该框架将大脑视为一个分层贝叶斯推理模型，通过反馈连接最小化预测误差。与使用反向传播（BP）训练的传统神经网络不同，PCNs采用推理学习（IL）——一种更具生物学合理性的算法，能够解释BP无法解释的神经活动模式。历史上，IL的计算量更大，但近期的进展表明，在充分并行化的情况下，IL可以实现比BP更高的效率。此外，从数学角度可将PCNs视为传统前馈神经网络（FNNs）的超集，从而极大地扩展了可训练架构的范围。作为固有的概率（图）隐变量模型，PCNs为监督学习和无监督（生成）建模提供了一个超越传统人工神经网络的通用框架。本文对PCNs进行了全面回顾和详细的形式化规范，特别将其置于现代机器学习方法的背景下进行阐述。这使预测编码成为一个有望推动未来机器学习创新的前景广阔的框架。