Recently, because of the high-quality representations of contrastive learning methods, rehearsal-based contrastive continual learning has been proposed to explore how to continually learn transferable representation embeddings to avoid the catastrophic forgetting issue in traditional continual settings. Based on this framework, we propose Contrastive Continual Learning via Importance Sampling (CCLIS) to preserve knowledge by recovering previous data distributions with a new strategy for Replay Buffer Selection (RBS), which minimize estimated variance to save hard negative samples for representation learning with high quality. Furthermore, we present the Prototype-instance Relation Distillation (PRD) loss, a technique designed to maintain the relationship between prototypes and sample representations using a self-distillation process. Experiments on standard continual learning benchmarks reveal that our method notably outperforms existing baselines in terms of knowledge preservation and thereby effectively counteracts catastrophic forgetting in online contexts. The code is available at https://github.com/lijy373/CCLIS.

翻译：摘要：最近，由于对比学习方法具有高质量的表征能力，基于回放的对比持续学习被提出，旨在探索如何持续学习可迁移的表征嵌入，从而避免传统持续学习场景中的灾难性遗忘问题。在此框架基础上，我们提出了一种基于重要性采样的对比持续学习（CCLIS）方法，通过一种新的回放缓冲区选择（RBS）策略恢复先前的数据分布，该策略最小化估计方差以保存难负样本，从而获得高质量的表征学习。此外，我们提出了原型-实例关系蒸馏（PRD）损失函数，这是一种利用自蒸馏过程维持原型与样本表征之间关系的技术。在标准持续学习基准上的实验表明，我们的方法在知识保留方面显著优于现有基线方法，从而有效抑制了在线场景中的灾难性遗忘。代码已开源在 https://github.com/lijy373/CCLIS。