Real-time recurrent learning (RTRL) computes exact online gradients by propagating a Jacobian tensor forward through recurrent dynamics, but at O(n^4) cost per step. Prior work has sought structured approximations (rank-1 compression, graph-based sparsity, Kronecker factorization). We show that, in the continuous error signal regime, the recurrent Jacobian is massively redundant:propagating through a random 6% of paths (k=4 of n=64) recovers 84 +/- 6% of full RTRL's adaptation ability across five seeds, and the absolute count k=4 remains effective from n=64 to n=256 (6% to 1.6%, recovery 84 to 78%), meaning sparse RTRL becomes relatively cheaper as networks grow. In RNNs, the recovery is selection-invariant (even adversarial path selection works) and exhibits a step-function transition from zero to any nonzero propagation. Spectral analysis reveals the mechanism: the Jacobian is full-rank but near-isotropic (condition numbers 2.6-6.5), so any random subset provides a directionally representative gradient estimate. On chaotic dynamics (Lorenz attractor), sparse propagation is more numerically stable than full RTRL (CV 13% vs. 88%), as subsampling avoids amplifying pathological spectral modes. The redundancy extends to LSTMs (k=4 matches full RTRL) and to transformers via sparse gradient transport (50% head sparsity outperforms the dense reference; 33% is borderline), with higher thresholds reflecting head specialization rather than isotropy. On real primate neural data, sparse RTRL (k=4) adapts online to cross-session electrode drift (80 +/- 11% recovery, 5 seeds), where sparse propagation is again more stable than full RTRL. Without continuous error signal, Jacobian propagation accumulates numerical drift and degrades all RTRL variants, a scope condition for all forward-mode methods. Results hold with SGD (92 +/- 1% recovery), suggesting independence from optimizer choice.

翻译：实时递归学习（RTRL）通过沿递归动力学向前传播雅可比张量来计算精确在线梯度，但每步计算代价为O(n^4)。先前的工作探索了结构化近似（秩-1压缩、基于图的稀疏性、Kronecker分解）。我们证明，在连续误差信号机制下，递归雅可比矩阵存在大规模冗余：通过随机6%的路径传播（n=64中k=4）可恢复全量RTRL适应能力的84±6%（五个随机种子），且固定计数k=4在n=64到n=256（6%到1.6%，恢复率84%到78%）范围内保持有效，这意味着随着网络规模增大，稀疏RTRL的相对计算成本更低。在RNN中，恢复具有选择不变性（甚至对抗性路径选择依然有效），并呈现从零传播到非零传播的阶跃函数式转变。谱分析揭示了其机制：雅可比矩阵满秩但近各向同性（条件数2.6-6.5），因此任意随机子集均可提供方向代表性的梯度估计。在混沌动力学（Lorenz吸引子）中，稀疏传播相比全量RTRL具有更高的数值稳定性（变异系数13%对比88%），因为子采样避免了放大病态谱模式。这种冗余性扩展到LSTM（k=4匹配全量RTRL）和Transformer（通过稀疏梯度传输，50%头部稀疏性超越密集参考模型，33%为临界值），其中更高阈值反映了头部特化而非各向同性。在真实灵长类神经数据上，稀疏RTRL（k=4）可在线适应跨会话电极漂移（恢复率80±11%，5个随机种子），且稀疏传播再次比全量RTRL更稳定。若缺乏连续误差信号，雅可比传播会积累数值漂移并降低所有RTRL变体的性能——这是所有前向模式方法的适用范围条件。结果在SGD优化器下成立（恢复率92±1%），表明与优化器选择无关。