Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.

翻译：预测编码（PC）为神经网络训练提供了一种受大脑启发的反向传播替代方案，被描述为最小化其内部能量的物理系统。然而，在实践中，PC主要采用数字模拟方式实现，需要大量计算资源且难以扩展至更深层架构。本文重新构建了PC以克服这种硬件-算法不匹配问题。首先，我们揭示了基于状态的经典PC公式（sPC）在设计上对数字模拟存在固有低效性，不可避免地导致指数级信号衰减，从而使整个最小化过程停滞。为克服这一根本性限制，我们引入了基于误差的PC（ePC）——一种新颖的PC重参数化方法，该方法不受信号衰减影响。尽管ePC不再具有生物学合理性，但其能数值计算精确的PC权重梯度，且运行速度比sPC快数个数量级。在多种架构和数据集上的实验表明，即使在sPC难以应对的深层模型中，ePC仍能匹配反向传播的性能。除实践改进外，本研究为PC动力学提供了理论见解，并为在数字硬件及其他平台上扩展基于PC的学习至更深层架构奠定了基础。