Predictive Coding (PC) is a biologically-inspired learning framework characterised by local, parallelisable operations, properties that enable energy-efficient implementation on neuromorphic hardware. Despite this, extending PC effectively to recurrent neural networks (RNNs) has been challenging, particularly for tasks involving long-range temporal dependencies. Backpropagation Through Time (BPTT) remains the dominant method for training RNNs, but its non-local computation, lack of spatial parallelism, and requirement to store extensive activation histories results in significant energy consumption. This work introduces a novel method combining Temporal Predictive Coding (tPC) with approximate Real-Time Recurrent Learning (RTRL), enabling effective spatio-temporal credit assignment. Results indicate that the proposed method can closely match the performance of BPTT on both synthetic benchmarks and real-world tasks. On a challenging machine translation task, with a 15-million parameter model, the proposed method achieves a test perplexity of 7.62 (vs. 7.49 for BPTT), marking one of the first applications of tPC to tasks of this scale. These findings demonstrate the potential of this method to learn complex temporal dependencies whilst retaining the local, parallelisable, and flexible properties of the original PC framework, paving the way for more energy-efficient learning systems.

翻译：预测编码（PC）是一种受生物学启发的学习框架，其特点是具有局部、可并行化的操作特性，这些特性使其能够在神经形态硬件上实现高能效。尽管如此，将PC有效扩展到循环神经网络（RNN）一直具有挑战性，尤其是在涉及长程时序依赖性的任务中。随时间反向传播（BPTT）仍然是训练RNN的主要方法，但其非局部计算、缺乏空间并行性以及需要存储大量激活历史的特点导致了显著的能耗。本研究提出了一种将时序预测编码（tPC）与近似实时循环学习（RTRL）相结合的新方法，实现了有效的时空信用分配。结果表明，所提出的方法在合成基准测试和实际任务上都能与BPTT的性能高度匹配。在一个具有挑战性的机器翻译任务中，使用一个1500万参数的模型，所提出的方法实现了7.62的测试困惑度（BPTT为7.49），这标志着tPC首次应用于此类规模的任务之一。这些发现证明了该方法在学习复杂时序依赖性的同时，保留了原始PC框架的局部性、可并行性和灵活性，为开发更节能的学习系统铺平了道路。