According to the Complementary Learning Systems (CLS) theory~\cite{mcclelland1995there} in neuroscience, humans do effective \emph{continual learning} through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics, individual experiences; and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment. Motivated by this theory, we propose \emph{DualNets} (for Dual Networks), a general continual learning framework comprising a fast learning system for supervised learning of pattern-separated representation from specific tasks and a slow learning system for representation learning of task-agnostic general representation via Self-Supervised Learning (SSL). DualNets can seamlessly incorporate both representation types into a holistic framework to facilitate better continual learning in deep neural networks. Via extensive experiments, we demonstrate the promising results of DualNets on a wide range of continual learning protocols, ranging from the standard offline, task-aware setting to the challenging online, task-free scenario. Notably, on the CTrL~\cite{veniat2020efficient} benchmark that has unrelated tasks with vastly different visual images, DualNets can achieve competitive performance with existing state-of-the-art dynamic architecture strategies~\cite{ostapenko2021continual}. Furthermore, we conduct comprehensive ablation studies to validate DualNets efficacy, robustness, and scalability. Code will be made available at \url{https://github.com/phquang/DualNet}.

翻译：根据神经科学中的互补学习系统理论，人类通过两个互补系统进行有效的持续学习：一个是以海马体为中心的快速学习系统，负责快速学习具体细节和个体经验；另一个是位于新皮质的缓慢学习系统，负责逐渐获取关于环境的结构化知识。受此理论启发，我们提出DualNets（双网络），一种通用的持续学习框架，包含一个快速学习系统，用于通过监督学习从特定任务中学习模式分离表示；以及一个慢速学习系统，通过自监督学习学习任务无关的通用表示。DualNets能无缝地将这两种表示类型整合到一个整体框架中，以促进深度神经网络中更好的持续学习。通过大量实验，我们在从标准的离线、任务感知场景到具有挑战性的在线、无任务场景的广泛持续学习协议中，展示了DualNets的显著成果。值得注意的是，在包含视觉图像差异巨大、任务不相关的CTrL基准测试中，DualNets能够与现有最先进的动态架构策略竞争。此外，我们进行了全面的消融研究，以验证DualNets的有效性、鲁棒性和可扩展性。代码将在 \url{https://github.com/phquang/DualNet} 提供。