Rehearsal, retraining on a stored small data subset of old tasks, has been proven effective in solving catastrophic forgetting in continual learning. However, due to the sampled data may have a large bias towards the original dataset, retraining them is susceptible to driving continual domain drift of old tasks in feature space, resulting in forgetting. In this paper, we focus on tackling the continual domain drift problem with centroid distance distillation. First, we propose a centroid caching mechanism for sampling data points based on constructed centroids to reduce the sample bias in rehearsal. Then, we present a centroid distance distillation that only stores the centroid distance to reduce the continual domain drift. The experiments on four continual learning datasets show the superiority of the proposed method, and the continual domain drift can be reduced.

翻译：回放，即在存储的旧任务小数据子集上进行重新训练，已被证明能有效解决持续学习中的灾难性遗忘问题。然而，由于采样数据可能对原始数据集存在较大偏差，重新训练这些数据易导致旧任务在特征空间中产生持续的领域漂移，从而引发遗忘。本文聚焦于通过质心距离蒸馏解决持续领域漂移问题。首先，我们提出一种基于构建质心的质心缓存机制来采样数据点，以降低回放中的样本偏差。随后，我们提出一种质心距离蒸馏方法，仅存储质心距离以减少持续的领域漂移。在四个持续学习数据集上的实验表明，所提方法具有优越性，且能够有效降低持续领域漂移。