Time-dependent data-generating distributions have proven to be difficult for gradient-based training of neural networks, as the greedy updates result in catastrophic forgetting of previously learned knowledge. Despite the progress in the field of continual learning to overcome this forgetting, we show that a set of common state-of-the-art methods still suffers from substantial forgetting upon starting to learn new tasks, except that this forgetting is temporary and followed by a phase of performance recovery. We refer to this intriguing but potentially problematic phenomenon as the stability gap. The stability gap had likely remained under the radar due to standard practice in the field of evaluating continual learning models only after each task. Instead, we establish a framework for continual evaluation that uses per-iteration evaluation and we define a new set of metrics to quantify worst-case performance. Empirically we show that experience replay, constraint-based replay, knowledge-distillation, and parameter regularization methods are all prone to the stability gap; and that the stability gap can be observed in class-, task-, and domain-incremental learning benchmarks. Additionally, a controlled experiment shows that the stability gap increases when tasks are more dissimilar. Finally, by disentangling gradients into plasticity and stability components, we propose a conceptual explanation for the stability gap.

翻译：随时间变化的数据生成分布已被证明对基于梯度的神经网络训练具有挑战性，因为贪婪的更新会导致先前所学知识的灾难性遗忘。尽管持续学习领域在克服这种遗忘方面取得了进展，但我们表明，一系列常见的先进方法在开始学习新任务时仍然会遭受显著的遗忘，只不过这种遗忘是暂时的，随后会出现性能恢复阶段。我们将这种有趣但可能存在问题现象称为稳定性差距。由于该领域仅在每个任务之后评估持续学习模型的常规做法，稳定性差距可能一直未被察觉。相反，我们建立了一个持续评估框架，该框架使用每轮迭代进行评估，并定义了一组新的度量标准来量化最坏情况下的性能。实验表明，经验回放、基于约束的回放、知识蒸馏和参数正则化方法都容易出现稳定性差距；并且稳定性差距可以在类增量、任务增量和域增量学习基准中观察到。此外，一项对照实验表明，当任务差异性更大时，稳定性差距会增大。最后，通过将梯度分解为可塑性和稳定性成分，我们提出了稳定性差距的概念性解释。