Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes. The key feature of CIL is that no task identifier (or task-id) is provided at test time for each test sample. Predicting the task-id for each test sample is a challenging problem. An emerging theoretically justified and effective approach is to train a task-specific model for each task in a shared network for all tasks based on a task-incremental learning (TIL) method to deal with forgetting. The model for each task in this approach is an out-of-distribution (OOD) detector rather than a conventional classifier. The OOD detector can perform both within-task (in-distribution (IND)) class prediction and OOD detection. The OOD detection capability is the key for task-id prediction during inference for each test sample. However, this paper argues that using a traditional OOD detector for task-id prediction is sub-optimal because additional information (e.g., the replay data and the learned tasks) available in CIL can be exploited to design a better and principled method for task-id prediction. We call the new method TPLR (Task-id Prediction based on Likelihood Ratio}). TPLR markedly outperforms strong CIL baselines.
翻译:类增量学习是持续学习中一个具有挑战性的场景,其需要按顺序学习一系列任务,每个任务包含一组独特的类别。类增量学习的关键特征在于测试阶段不为每个测试样本提供任务标识符(即任务ID),因此预测每个测试样本的任务ID成为一个难题。一种新兴且具有理论依据的有效方法基于任务增量学习方法,在共享网络中为每个任务训练一个专用模型以应对遗忘问题。该方法的每个任务模型本质上是分布外检测器而非传统分类器。这种分布外检测器既能执行任务内(分布内)类别预测,也能进行分布外检测。在推理阶段,分布外检测能力是预测每个测试样本任务ID的关键。然而,本文论证了使用传统分布外检测器进行任务ID预测是次优的,因为类增量学习场景中可利用额外信息(如重放数据与已学习任务)来设计更优且更规范的任务ID预测方法。我们将新方法命名为TPLR(基于似然比的任务ID预测)。TPLR显著优于强基线类增量学习方法。