Class incremental learning(CIL) has attracted much attention, but most existing related works focus on fine-tuning the entire representation model, which inevitably results in much catastrophic forgetting. In the contrast, with a semantic-rich pre-trained representation model, parameter-additional-tuning (PAT) only changes very few parameters to learn new visual concepts. Recent studies have proved that PAT-based CIL can naturally avoid fighting against forgetting by replaying or distilling like most of the existing methods. However, we find that PAT-based CIL still faces serious semantic drift, the high-level forgetting problem caused by classifier learning bias at different learning phases, which significantly reduces the performance of PAT-based CIL. To address this problem, we propose Incremental Prototype Tuning (IPT), a simple but effective method that tunes category prototypes for classification and learning example prototypes to compensate for semantic drift. Extensive experiments demonstrate that our method can effectively compensate for semantic drift. Combined with well-pre-trained Vit backbones and other PAT methods, IPT surpasses the state-of-the-art baselines on mainstream incremental learning benchmarks.

翻译：类增量学习（CIL）近年来受到广泛关注，但现有相关工作大多聚焦于微调整个表征模型，这不可避免地导致严重的灾难性遗忘。相比之下，基于语义丰富的预训练表征模型，参数附加调优（PAT）仅需改变极少量参数即可学习新的视觉概念。近期研究证明，基于PAT的CIL方法能够天然地避免像多数现有方法那样通过回放或蒸馏来对抗遗忘。然而，我们发现基于PAT的CIL仍面临严重的语义漂移问题——这是由不同学习阶段分类器学习偏差引发的高层次遗忘问题，会显著降低PAT-CIL的性能。为解决该问题，我们提出增量式原型调优（IPT），这是一种简单而有效的方法：通过调优类别原型进行分类，并学习示例原型以补偿语义漂移。大量实验表明，我们的方法能有效补偿语义漂移。与预训练良好的ViT主干网络及其他PAT方法相结合，IPT在主流增量学习基准上超越了当前最先进的基线方法。