Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control problem where learning and preservation signals compete within neural activity dynamics. We convert regularization penalties into preservation signals that protect prior-task representations. Learning then proceeds by minimizing the control effort required to integrate new tasks while competing with the preservation of prior tasks. At equilibrium, the neural activities produce weight updates that implicitly encode the full prior-task curvature, a property we term the continual-natural gradient, requiring no explicit curvature storage. Experiments confirm that our learning framework recovers true prior-task curvature and enables task discrimination, outperforming existing methods on standard benchmarks without replay.

翻译：灾难性遗忘仍然是神经网络在顺序学习任务时面临的根本挑战。在本研究中，我们将持续学习重新构建为控制问题，其中学习信号与保持信号在神经活动动态中相互竞争。我们将正则化惩罚项转化为保持信号，用以保护先前任务的表征。随后通过最小化整合新任务所需控制力（同时与先前任务保持相竞争）来实现学习过程。在平衡状态下，神经活动产生的权重更新隐式编码了完整的先前任务曲率——这一特性我们称之为持续自然梯度，无需显式存储曲率信息。实验证实，我们的学习框架能够恢复真实的先前任务曲率并实现任务区分，在标准基准测试中无需重放机制即可超越现有方法。