In real-world applications, dynamic scenarios require the models to possess the capability to learn new tasks continuously without forgetting the old knowledge. Experience-Replay methods store a subset of the old images for joint training. In the scenario of more strict privacy protection, storing the old images becomes infeasible, which leads to a more severe plasticity-stability dilemma and classifier bias. To meet the above challenges, we propose a new architecture, named continual expansion and absorption transformer~(CEAT). The model can learn the novel knowledge by extending the expanded-fusion layers in parallel with the frozen previous parameters. After the task ends, we losslessly absorb the extended parameters into the backbone to ensure that the number of parameters remains constant. To improve the learning ability of the model, we designed a novel prototype contrastive loss to reduce the overlap between old and new classes in the feature space. Besides, to address the classifier bias towards the new classes, we propose a novel approach to generate the pseudo-features to correct the classifier. We experiment with our methods on three standard Non-Exemplar Class-Incremental Learning~(NECIL) benchmarks. Extensive experiments demonstrate that our model gets a significant improvement compared with the previous works and achieves 5.38%, 5.20%, and 4.92% improvement on CIFAR-100, TinyImageNet, and ImageNet-Subset.

翻译：在实际应用中，动态场景要求模型具备持续学习新任务且不遗忘旧知识的能力。经验回放方法通过存储部分旧图像进行联合训练，但在隐私保护更严格的场景下，存储旧图像变得不可行，进而导致更严重的塑性-稳定性困境及分类器偏差。为应对上述挑战，我们提出了一种名为持续扩展与吸收Transformer（CEAT）的新型架构。该模型通过在与冻结的先前参数平行的位置扩展融合层来学习新知识，并在任务结束后将扩展参数无损吸收回主干网络，确保参数量保持恒定。为提升模型学习能力，我们设计了一种新型原型对比损失函数，以降低特征空间中新旧类别的重叠。此外，针对分类器偏向新类别的问题，我们提出了一种生成伪特征来校正分类器的新方法。我们在三个标准无样本类增量学习（NECIL）基准上进行了实验。大量实验表明，与先前工作相比，我们的模型取得了显著改进，在CIFAR-100、TinyImageNet和ImageNet-Subset上分别实现了5.38%、5.20%和4.92%的性能提升。