Most continual segmentation methods tackle the problem as a per-pixel classification task. However, such a paradigm is very challenging, and we find query-based segmenters with built-in objectness have inherent advantages compared with per-pixel ones, as objectness has strong transfer ability and forgetting resistance. Based on these findings, we propose CoMasTRe by disentangling continual segmentation into two stages: forgetting-resistant continual objectness learning and well-researched continual classification. CoMasTRe uses a two-stage segmenter learning class-agnostic mask proposals at the first stage and leaving recognition to the second stage. During continual learning, a simple but effective distillation is adopted to strengthen objectness. To further mitigate the forgetting of old classes, we design a multi-label class distillation strategy suited for segmentation. We assess the effectiveness of CoMasTRe on PASCAL VOC and ADE20K. Extensive experiments show that our method outperforms per-pixel and query-based methods on both datasets. Code will be available at https://github.com/jordangong/CoMasTRe.

翻译：大多数持续分割方法将问题视为逐像素分类任务。然而，这种范式极具挑战性，我们发现具备内置对象性（objectness）的查询式分割器相比逐像素方法具有固有优势，因为对象性具有强迁移能力和抗遗忘特性。基于这些发现，我们提出CoMasTRe方法，将持续分割解耦为两个阶段：抗遗忘的持续对象性学习与成熟的持续分类。CoMasTRe采用两阶段分割器，第一阶段学习类别无关的掩码提案，第二阶段负责识别任务。在持续学习过程中，采用简洁而有效的蒸馏策略强化对象性。为进一步缓解旧类别的遗忘问题，我们设计了适用于分割任务的多标签类别蒸馏策略。我们在PASCAL VOC和ADE20K数据集上验证了CoMasTRe的有效性。大量实验表明，我们的方法在两个数据集上均优于逐像素方法和查询式方法。代码将发布于https://github.com/jordangong/CoMasTRe。