In contrast to batch learning where all training data is available at once, continual learning represents a family of methods that accumulate knowledge and learn continuously with data available in sequential order. Similar to the human learning process with the ability of learning, fusing, and accumulating new knowledge coming at different time steps, continual learning is considered to have high practical significance. Hence, continual learning has been studied in various artificial intelligence tasks. In this paper, we present a comprehensive review of the recent progress of continual learning in computer vision. In particular, the works are grouped by their representative techniques, including regularization, knowledge distillation, memory, generative replay, parameter isolation, and a combination of the above techniques. For each category of these techniques, both its characteristics and applications in computer vision are presented. At the end of this overview, several subareas, where continuous knowledge accumulation is potentially helpful while continual learning has not been well studied, are discussed.

翻译：与所有训练数据一次性可用的批量学习不同，持续学习是一系列通过按序获取数据、持续积累知识并学习的方法。类似于人类学习过程——具备在不同时间节点学习、融合和积累新知识的能力，持续学习被认为具有高度的实际意义。因此，持续学习已在各种人工智能任务中得到研究。本文全面回顾了计算机视觉领域持续学习的最新进展。具体而言，相关研究工作按其代表性技术分类，包括正则化、知识蒸馏、记忆、生成重放、参数隔离以及上述技术的组合。针对每一类技术，我们既阐述其特性，也介绍其在计算机视觉中的应用。在本综述末尾，我们讨论了若干子领域——这些领域中持续知识积累具有潜在价值，但持续学习尚未得到充分研究。