Backpropagation has rapidly become the workhorse credit assignment algorithm for modern deep learning methods. Recently, modified forms of predictive coding (PC), an algorithm with origins in computational neuroscience, have been shown to result in approximately or exactly equal parameter updates to those under backpropagation. Due to this connection, it has been suggested that PC can act as an alternative to backpropagation with desirable properties that may facilitate implementation in neuromorphic systems. Here, we explore these claims using the different contemporary PC variants proposed in the literature. We obtain time complexity bounds for these PC variants which we show are lower-bounded by backpropagation. We also present key properties of these variants that have implications for neurobiological plausibility and their interpretations, particularly from the perspective of standard PC as a variational Bayes algorithm for latent probabilistic models. Our findings shed new light on the connection between the two learning frameworks and suggest that, in its current forms, PC may have more limited potential as a direct replacement of backpropagation than previously envisioned.

翻译：反向传播算法已成为现代深度学习方法中不可或缺的信用分配工作流算法。近年来，源于计算神经科学的预测编码（PC）算法的改良形式被证实在参数更新上与反向传播近似或完全等价。基于此关联，学者提出PC可作为具有理想特性的反向传播替代方案，尤其利于神经形态系统的实现。本文通过系统梳理文献中提出的不同当代PC变体，深入探究上述论断，推导出各变体的时间复杂度界限，并证明其均受限于反向传播的复杂度下限。同时揭示这些变体在神经生物学合理性及其解释上的关键特性，特别从标准PC作为隐概率模型变分贝叶斯算法的视角进行阐释。本研究表明，两种学习框架间的关联性比此前认知更为复杂，在现有形式下，PC作为反向传播直接替代方案的潜力可能比预期更为有限。