Continual learning (CL) models are designed to learn new tasks arriving sequentially without re-training the network. However, real-world ML applications have very limited label information and these models suffer from catastrophic forgetting. To address these issues, we propose an unsupervised CL model with task experts called Unsupervised Task Expert Lifelong Learning (U-TELL) to continually learn the data arriving in a sequence addressing catastrophic forgetting. During training of U-TELL, we introduce a new expert on arrival of a new task. Our proposed architecture has task experts, a structured data generator and a task assigner. Each task expert is composed of 3 blocks; i) a variational autoencoder to capture the task distribution and perform data abstraction, ii) a k-means clustering module, and iii) a structure extractor to preserve latent task data signature. During testing, task assigner selects a suitable expert to perform clustering. U-TELL does not store or replay task samples, instead, we use generated structured samples to train the task assigner. We compared U-TELL with five SOTA unsupervised CL methods. U-TELL outperformed all baselines on seven benchmarks and one industry dataset for various CL scenarios with a training time over 6 times faster than the best performing baseline.

翻译：持续学习（CL）模型旨在顺序学习新到达的任务，而无需重新训练网络。然而，现实世界的机器学习应用标签信息非常有限，且这些模型存在灾难性遗忘问题。为解决这些问题，我们提出了一种基于任务专家的无监督持续学习模型——无监督任务专家终身学习（U-TELL），以持续学习顺序到达的数据并应对灾难性遗忘。在U-TELL训练过程中，每当新任务到达时，我们引入一个新的专家。所提出的架构包含任务专家、结构化数据生成器和任务分配器。每个任务专家由三个模块组成：i）变分自编码器，用于捕获任务分布并执行数据抽象；ii）k-means聚类模块；iii）结构提取器，用于保留潜在任务数据特征。在测试阶段，任务分配器选择合适的专家执行聚类。U-TELL不存储或重放任务样本，而是使用生成的结构化样本来训练任务分配器。我们将U-TELL与五种最先进的无监督持续学习方法进行了比较。在七个基准数据集和一个行业数据集上，U-TELL在各种持续学习场景中均优于所有基线方法，且训练时间比表现最佳的基线方法快6倍以上。