In this work, we investigate exemplar-free class incremental learning (CIL) with knowledge distillation (KD) as a regularization strategy, aiming to prevent forgetting. KD-based methods are successfully used in CIL, but they often struggle to regularize the model without access to exemplars of the training data from previous tasks. Our analysis reveals that this issue originates from substantial representation shifts in the teacher network when dealing with out-of-distribution data. This causes large errors in the KD loss component, leading to performance degradation in CIL models. Inspired by recent test-time adaptation methods, we introduce Teacher Adaptation (TA), a method that concurrently updates the teacher and the main models during incremental training. Our method seamlessly integrates with KD-based CIL approaches and allows for consistent enhancement of their performance across multiple exemplar-free CIL benchmarks.

翻译：本文研究无样本类增量学习（CIL）中，以知识蒸馏（KD）作为正则化策略来防止遗忘的问题。基于KD的方法在CIL中已成功应用，但缺乏先前任务训练数据的样本时，往往难以有效正则化模型。我们的分析表明，该问题源于教师网络在处理分布外数据时出现的显著表征偏移，这导致KD损失分量产生较大误差，进而造成CIL模型的性能退化。受近期测试时自适应方法的启发，我们提出教师自适应（TA）方法——在增量训练过程中同步更新教师模型与主模型。该方法能无缝集成至基于KD的CIL框架，并在多个无样本CIL基准测试中持续提升其性能。