Over the past decade, deep neural networks have demonstrated significant success using the training scheme that involves mini-batch stochastic gradient descent on extensive datasets. Expanding upon this accomplishment, there has been a surge in research exploring the application of neural networks in other learning scenarios. One notable framework that has garnered significant attention is meta-learning. Often described as "learning to learn," meta-learning is a data-driven approach to optimize the learning algorithm. Other branches of interest are continual learning and online learning, both of which involve incrementally updating a model with streaming data. While these frameworks were initially developed independently, recent works have started investigating their combinations, proposing novel problem settings and learning algorithms. However, due to the elevated complexity and lack of unified terminology, discerning differences between the learning frameworks can be challenging even for experienced researchers. To facilitate a clear understanding, this paper provides a comprehensive survey that organizes various problem settings using consistent terminology and formal descriptions. By offering an overview of these learning paradigms, our work aims to foster further advancements in this promising area of research.

翻译：过去十年间，深度神经网络利用基于小批量随机梯度下降的大规模数据集训练方案取得了显著成功。在此成就基础上，涌现出大量探索神经网络在其他学习场景中应用的研究。其中，元学习作为备受瞩目的框架之一，常被称为"学习如何学习"，是一种通过数据驱动方式优化学习算法的方法。其他值得关注的领域包括持续学习和在线学习，两者均涉及利用流式数据增量更新模型。尽管这些框架最初是独立发展的，但近期研究已开始探索其组合应用，提出了新颖的问题设置与学习算法。然而，由于复杂性增加且缺乏统一术语，即便经验丰富的研究者也可能难以辨别不同学习框架间的差异。为促进清晰理解，本文提供了一份全面综述，采用一致术语与形式化描述对各种问题设置进行系统梳理。通过概述这些学习范式，我们的工作旨在推动这一前景广阔的研究领域取得进一步进展。