Continual Learning (CL) focuses on learning from dynamic and changing data distributions while retaining previously acquired knowledge. Various methods have been developed to address the challenge of catastrophic forgetting, including regularization-based, Bayesian-based, and memory-replay-based techniques. However, these methods lack a unified framework and common terminology for describing their approaches. This research aims to bridge this gap by introducing a comprehensive and overarching framework that encompasses and reconciles these existing methodologies. Notably, this new framework is capable of encompassing established CL approaches as special instances within a unified and general optimization objective. An intriguing finding is that despite their diverse origins, these methods share common mathematical structures. This observation highlights the compatibility of these seemingly distinct techniques, revealing their interconnectedness through a shared underlying optimization objective. Moreover, the proposed general framework introduces an innovative concept called refresh learning, specifically designed to enhance the CL performance. This novel approach draws inspiration from neuroscience, where the human brain often sheds outdated information to improve the retention of crucial knowledge and facilitate the acquisition of new information. In essence, refresh learning operates by initially unlearning current data and subsequently relearning it. It serves as a versatile plug-in that seamlessly integrates with existing CL methods, offering an adaptable and effective enhancement to the learning process. Extensive experiments on CL benchmarks and theoretical analysis demonstrate the effectiveness of the proposed refresh learning. Code is available at \url{https://github.com/joey-wang123/CL-refresh-learning}.

翻译：持续学习（CL）专注于从动态变化的数据分布中学习，同时保留先前获取的知识。针对灾难性遗忘这一挑战，研究者已开发出多种方法，包括基于正则化、贝叶斯方法和记忆重放的技术。然而，这些方法缺乏统一的框架和通用术语来描述其方法。本研究旨在通过引入一个全面且覆盖广泛的框架，整合并调合这些现有方法，从而弥合这一差距。值得注意的是，该新框架能够将已有的CL方法视为统一通用优化目标中的特例。一个有趣的发现是，尽管这些方法来源各异，但它们共享共同的数学结构。这一观察结果突显了这些看似不同技术的兼容性，揭示了它们通过共享底层优化目标而相互关联。此外，该通用框架引入了一个名为刷新学习（refresh learning）的创新概念，专门用于提升CL性能。这一新方法受神经科学启发，人类大脑常通过摒弃过时信息来增强关键知识的保留并促进新知识的获取。本质上，刷新学习的运作方式是先遗忘当前数据，随后重新学习。它作为一个多功能插件，可无缝集成到现有CL方法中，为学习过程提供灵活且有效的增强。在CL基准测试上的广泛实验和理论分析证明了所提刷新学习的有效性。代码可在 \url{https://github.com/joey-wang123/CL-refresh-learning} 获取。