Unsupervised Continual Learning (UCL) is a burgeoning field in machine learning, focusing on enabling neural networks to sequentially learn tasks without explicit label information. Catastrophic Forgetting (CF), where models forget previously learned tasks upon learning new ones, poses a significant challenge in continual learning, especially in UCL, where labeled information of data is not accessible. CF mitigation strategies, such as knowledge distillation and replay buffers, often face memory inefficiency and privacy issues. Although current research in UCL has endeavored to refine data representations and address CF in streaming data contexts, there is a noticeable lack of algorithms specifically designed for unsupervised clustering. To fill this gap, in this paper, we introduce the concept of Unsupervised Continual Clustering (UCC). We propose Forward-Backward Knowledge Distillation for unsupervised Continual Clustering (FBCC) to counteract CF within the context of UCC. FBCC employs a single continual learner (the ``teacher'') with a cluster projector, along with multiple student models, to address the CF issue. The proposed method consists of two phases: Forward Knowledge Distillation, where the teacher learns new clusters while retaining knowledge from previous tasks with guidance from specialized student models, and Backward Knowledge Distillation, where a student model mimics the teacher's behavior to retain task-specific knowledge, aiding the teacher in subsequent tasks. FBCC marks a pioneering approach to UCC, demonstrating enhanced performance and memory efficiency in clustering across various tasks, outperforming the application of clustering algorithms to the latent space of state-of-the-art UCL algorithms.

翻译：无监督持续学习是机器学习中一个新兴领域，其核心在于使神经网络能够在没有显式标签信息的情况下顺序学习任务。灾难性遗忘——即模型在学习新任务时遗忘先前习得任务的现象——是持续学习尤其是无监督持续学习中的重大挑战，因为此时无法获取数据的标签信息。缓解灾难性遗忘的策略（如知识蒸馏和回放缓冲区）常面临内存效率低下和隐私问题。尽管当前无监督持续学习的研究致力于优化数据表示并解决流数据场景下的灾难性遗忘，但专门针对无监督聚类设计的算法仍明显缺乏。为填补这一空白，本文提出了无监督持续聚类的概念。我们针对无监督持续聚类场景设计了前向-后向知识蒸馏方法以应对灾难性遗忘。该方法采用单一持续学习器（即“教师”模型）配合聚类投影器，并结合多个学生模型来解决灾难性遗忘问题。所提方法包含两个阶段：前向知识蒸馏阶段，教师模型在专用学生模型指导下学习新聚类的同时保持对先前任务知识的掌握；后向知识蒸馏阶段，学生模型通过模仿教师行为来保留任务特定知识，以协助教师模型在后续任务中的学习。该方法是针对无监督持续聚类的开创性解决方案，在跨任务聚类中展现出优越的性能和内存效率，其表现超越了在先进无监督持续学习算法潜在空间直接应用聚类方法的效果。