Predictive Coding (PC) offers a biologically plausible alternative to backpropagation for neural network training, yet struggles with deeper architectures. This paper identifies the root cause and provides a principled solution. We uncover that the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient on digital hardware, due to an inherent signal decay problem that scales exponentially with depth. To address this fundamental limitation, we introduce a novel reparameterization of PC, named error-based PC (ePC), which does not suffer from signal decay. By optimizing over prediction errors rather than states, ePC enables signals to reach all layers simultaneously and unattenuated, converging orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling bio-inspired learning to deeper architectures on digital hardware and beyond.

翻译：预测编码（PC）为神经网络训练提供了一种生物学上可信的反向传播替代方案，但在更深层的架构中面临困难。本文揭示了其根本原因并提出了一个原则性的解决方案。我们发现，经典的基于状态的PC公式（sPC）在设计上存在固有的信号衰减问题，该问题随网络深度呈指数级增长，导致其在数字硬件上效率极低。为解决这一根本性限制，我们引入了一种新颖的PC重参数化方法，称为基于误差的PC（ePC），它不受信号衰减的影响。通过优化预测误差而非状态，ePC使信号能够同时且无衰减地到达所有层，其收敛速度比sPC快数个数量级。在多种架构和数据集上的实验表明，即使在sPC难以应对的更深层模型中，ePC也能达到与反向传播相当的性能。除了实际性能的提升，我们的工作为理解PC动态提供了理论见解，并为在数字硬件及其他平台上将受生物启发的学习扩展到更深层架构奠定了基础。