In real-world applications, dynamic scenarios require the models to possess the capability to learn new tasks continuously without forgetting the old knowledge. Experience-Replay methods store a subset of the old images for joint training. In the scenario of more strict privacy protection, storing the old images becomes infeasible, which leads to a more severe plasticity-stability dilemma and classifier bias. To meet the above challenges, we propose a new architecture, named continual expansion and absorption transformer~(CEAT). The model can learn the novel knowledge by extending the expanded-fusion layers in parallel with the frozen previous parameters. After the task ends, we losslessly absorb the extended parameters into the backbone to ensure that the number of parameters remains constant. To improve the learning ability of the model, we designed a novel prototype contrastive loss to reduce the overlap between old and new classes in the feature space. Besides, to address the classifier bias towards the new classes, we propose a novel approach to generate the pseudo-features to correct the classifier. We experiment with our methods on three standard Non-Exemplar Class-Incremental Learning~(NECIL) benchmarks. Extensive experiments demonstrate that our model gets a significant improvement compared with the previous works and achieves 5.38%, 5.20%, and 4.92% improvement on CIFAR-100, TinyImageNet, and ImageNet-Subset.

翻译：在现实应用场景中，动态环境要求模型具备在不遗忘旧知识的前提下持续学习新任务的能力。经验回放方法通过存储部分旧图像进行联合训练，但在隐私保护更为严格的情况下，存储旧图像已不可行，这导致模型面临更严重的可塑性-稳定性困境及分类器偏差问题。为应对上述挑战，我们提出一种名为持续扩展与吸收Transformer（CEAT）的新型架构。该模型通过在与冻结的先前参数并行的结构中扩展融合层来学习新知识，任务结束后将扩展参数无损吸收至主干网络，以保持参数量恒定。为提升模型学习能力，我们设计了新颖的原型对比损失函数，用以减少新旧类别在特征空间中的重叠。此外，针对分类器偏向新类别的问题，我们提出了一种生成伪特征校正分类器的新方法。我们在三个标准无样例类增量学习（NECIL）基准上开展实验，大量实验结果表明，与先前工作相比，我们的模型获得显著提升，在CIFAR-100、TinyImageNet和ImageNet-Subset数据集上分别实现了5.38%、5.20%和4.92%的性能提升。