Class-incremental learning (CIL) aims to adapt to emerging new classes without forgetting old ones. Traditional CIL models are trained from scratch to continually acquire knowledge as data evolves. Recently, pre-training has achieved substantial progress, making vast pre-trained models (PTMs) accessible for CIL. Contrary to traditional methods, PTMs possess generalizable embeddings, which can be easily transferred. In this work, we revisit CIL with PTMs and argue that the core factors in CIL are adaptivity for model updating and generalizability for knowledge transferring. 1) We first reveal that frozen PTM can already provide generalizable embeddings for CIL. Surprisingly, a simple baseline (SimpleCIL) which continually sets the classifiers of PTM to prototype features can beat state-of-the-art even without training on the downstream task. 2) Due to the distribution gap between pre-trained and downstream datasets, PTM can be further cultivated with adaptivity via model adapting. We propose ADapt And Merge (ADAM), which aggregates the embeddings of PTM and adapted models for classifier construction. ADAM is a general framework that can be orthogonally combined with any parameter-efficient tuning method, which holds the advantages of PTM's generalizability and adapted model's adaptivity. 3) Additionally, we find previous benchmarks are unsuitable in the era of PTM due to data overlapping and propose four new benchmarks for assessment, namely ImageNet-A, ObjectNet, OmniBenchmark, and VTAB. Extensive experiments validate the effectiveness of ADAM with a unified and concise framework.

翻译：类增量学习（CIL）旨在适应新出现的类别而不遗忘旧类别。传统CIL模型从零开始训练，随数据演化持续获取知识。近年来，预训练技术取得了显著进展，使得大量预训练模型（PTM）可用于CIL。与传统方法不同，PTM拥有易于迁移的泛化性嵌入。在本研究中，我们重新审视基于PTM的CIL，并论证CIL的核心因素在于模型更新的适应性和知识迁移的泛化能力。1）我们首先揭示，即使冻结的PTM也能为CIL提供泛化性嵌入。令人惊讶的是，一个简单的基线方法（SimpleCIL）——持续将PTM的分类器设置为原型特征——甚至无需在下游任务上训练即可超越现有最优方法。2）由于预训练数据集与下游数据集之间存在分布差异，可通过模型适配进一步提升PTM的适应性。我们提出适配与合并（ADAM）方法，聚合PTM和适配模型的嵌入以构建分类器。ADAM是一个通用框架，可与任意参数高效微调方法正交结合，兼容PTM的泛化能力与适配模型的适应性优势。3）此外，我们发现现有基准在PTM时代因数据重叠而不再适用，因此提出四个新评估基准：ImageNet-A、ObjectNet、OmniBenchmark及VTAB。大量实验验证了ADAM在统一简洁框架下的有效性。