Despite the success of deep learning methods on instance segmentation, these models still suffer from catastrophic forgetting in continual learning scenarios. In this paper, our contributions for continual instance segmentation are threefold. First, we propose the Y-knowledge distillation (Y-KD), a knowledge distillation strategy that shares a common feature extractor between the teacher and student networks. As the teacher is also updated with new data in Y-KD, the increased plasticity results in new modules that are specialized on new classes. Second, our Y-KD approach is supported by a dynamic architecture method that grows new modules for each task and uses all of them for inference with a unique instance segmentation head, which significantly reduces forgetting. Third, we complete our approach by leveraging checkpoint averaging as a simple method to manually balance the trade-off between the performance on the various sets of classes, thus increasing the control over the model's behavior without any additional cost. These contributions are united in our model that we name the Dynamic Y-KD network. We perform extensive experiments on several single-step and multi-steps scenarios on Pascal-VOC, and we show that our approach outperforms previous methods both on past and new classes. For instance, compared to recent work, our method obtains +2.1% mAP on old classes in 15-1, +7.6% mAP on new classes in 19-1 and reaches 91.5% of the mAP obtained by joint-training on all classes in 15-5.

翻译：尽管深度学习方法在实例分割任务上取得了成功，但在连续学习场景中，这些模型仍会遭受灾难性遗忘。本文针对连续实例分割的贡献分为三个方面。首先，我们提出Y-知识蒸馏（Y-KD），一种在教师网络与学生网络之间共享通用特征提取器的知识蒸馏策略。由于Y-KD中教师网络也会随新数据进行更新，其增强的可塑性使得新模块能够专门针对新类别进行学习。其次，我们的Y-KD方法通过动态架构方法得到支持，该方法为每个任务生成新模块，并使用独特实例分割头将所有模块用于推理，从而显著减少遗忘。最后，我们通过采用检查点平均方法完善方案，作为手动平衡各类别集合性能权衡的简单手段，从而在不增加额外成本的情况下提升对模型行为的控制。这些贡献统一于我们命名为动态Y-KD网络的模型中。我们在Pascal-VOC数据集上针对多个单步和多步场景进行了大量实验，结果表明我们的方法在旧类别和新类别上的表现均优于现有方法。例如，与近期工作相比，我们的方法在15-1场景中旧类别的平均精度（mAP）提升2.1%，在19-1场景中新类别的平均精度提升7.6%，并在15-5场景中达到所有类别联合训练所得平均精度的91.5%。