Class-incremental learning (CIL) is a challenging task that involves sequentially learning to categorize classes from new tasks without forgetting previously learned information. The advent of large pre-trained models (PTMs) has fast-tracked the progress in CIL due to the highly transferable PTM representations, where tuning a small set of parameters leads to state-of-the-art performance when compared with the traditional CIL methods that are trained from scratch. However, repeated fine-tuning on each task destroys the rich representations of the PTMs and further leads to forgetting previous tasks. To strike a balance between the stability and plasticity of PTMs for CIL, we propose a novel perspective of eliminating training on every new task and instead train PTM only on the first task, and then refine its representation at inference time using test-time adaptation (TTA). Concretely, we propose Test-Time Adaptation for Class-Incremental Learning (TTACIL) that first fine-tunes PTMs using Adapters on the first task, then adjusts Layer Norm parameters of the PTM on each test instance for learning task-specific features, and finally resets them back to the adapted model to preserve stability. As a consequence, our TTACIL does not undergo any forgetting, while benefiting each task with the rich PTM features. Additionally, by design, our TTACIL is robust to common data corruptions. Our method outperforms several state-of-the-art CIL methods when evaluated on multiple CIL benchmarks under both clean and corrupted data. Code is available at: https://github.com/IemProg/TTACIL.

翻译：类增量学习（CIL）是一项具有挑战性的任务，它涉及在顺序学习新任务类别的同时避免遗忘先前学到的信息。大规模预训练模型（PTMs）的出现因PTM具有高度可迁移的表征而加速了CIL的发展——与传统从头训练的CIL方法相比，仅需调整少量参数即可达到最先进的性能。然而，在每个任务上重复微调会破坏PTM的丰富表征，并进一步导致对先前任务的遗忘。为平衡PTM在CIL中的稳定性与可塑性，我们提出一种全新视角：消除在每个新任务上的训练过程，转而仅在第一个任务上训练PTM，然后在推理阶段通过测试时自适应（TTA）精炼其表征。具体而言，我们提出面向类增量学习的测试时自适应方法（TTACIL），该方法首先在第一个任务上使用适配器（Adapter）微调PTM，随后在每个测试样本上调整PTM的层归一化参数以学习任务特定特征，最后将这些参数恢复至已适配模型以保持稳定性。因此，我们的TTACIL在完全避免遗忘的同时，使每个任务都能受益于PTM的丰富特征。此外，按设计原理，TTACIL对常见数据损坏具有鲁棒性。在多个CIL基准测试（包含干净数据与受损数据）上的评估表明，我们的方法优于多个最先进的CIL方法。代码已开源：https://github.com/IemProg/TTACIL。