Deep neural networks have shown remarkable performance when trained on independent and identically distributed data from a fixed set of classes. However, in real-world scenarios, it can be desirable to train models on a continuous stream of data where multiple classification tasks are presented sequentially. This scenario, known as Continual Learning (CL) poses challenges to standard learning algorithms which struggle to maintain knowledge of old tasks while learning new ones. This stability-plasticity dilemma remains central to CL and multiple metrics have been proposed to adequately measure stability and plasticity separately. However, none considers the increasing difficulty of the classification task, which inherently results in performance loss for any model. In that sense, we analyze some limitations of current metrics and identify the presence of setup-induced forgetting. Therefore, we propose new metrics that account for the task's increasing difficulty. Through experiments on benchmark datasets, we demonstrate that our proposed metrics can provide new insights into the stability-plasticity trade-off achieved by models in the continual learning environment.

翻译：深度神经网络在固定类别集合上的独立同分布数据训练时表现出显著性能。然而，在现实场景中，通常需要在连续数据流上训练模型，其中多个分类任务按顺序呈现。这种被称为持续学习(CL)的场景对标准学习算法提出了挑战，这些算法在学习新任务时难以保持旧任务的知识。这种稳定性-可塑性困境仍是CL的核心问题，已有多个指标被提出以分别充分衡量稳定性和可塑性。然而，现有指标均未考虑分类任务日益增长的难度——这本身就必然导致任何模型性能的损失。基于此，我们分析了当前指标的部分局限性，并识别了设置诱导遗忘的存在。因此，我们提出了考虑任务难度增加的新指标。通过在基准数据集上的实验，我们证明所提出的指标能够为持续学习环境下模型实现的稳定性-可塑性权衡提供新的见解。