Unsupervised lifelong learning refers to the ability to learn over time while memorizing previous patterns without supervision. Although great progress has been made in this direction, existing work often assumes strong prior knowledge about the incoming data (e.g., knowing the class boundaries), which can be impossible to obtain in complex and unpredictable environments. In this paper, motivated by real-world scenarios, we propose a more practical problem setting called online self-supervised lifelong learning without prior knowledge. The proposed setting is challenging due to the non-iid and single-pass data, the absence of external supervision, and no prior knowledge. To address the challenges, we propose Self-Supervised ContrAstive Lifelong LEarning without Prior Knowledge (SCALE) which can extract and memorize representations on the fly purely from the data continuum. SCALE is designed around three major components: a pseudo-supervised contrastive loss, a self-supervised forgetting loss, and an online memory update for uniform subset selection. All three components are designed to work collaboratively to maximize learning performance. We perform comprehensive experiments of SCALE under iid and four non-iid data streams. The results show that SCALE outperforms the state-of-the-art algorithm in all settings with improvements up to 3.83%, 2.77% and 5.86% in terms of kNN accuracy on CIFAR-10, CIFAR-100, and TinyImageNet datasets.

翻译：无监督终身学习指在无需监督的情况下随时间学习并记忆先前模式的能力。尽管该方向已取得显著进展，现有工作通常假设对输入数据具备强先验知识（例如知晓类别边界），这在复杂不可预测的环境中往往难以获取。本文基于真实场景需求，提出一种更具实用性的问题设定——无先验知识的在线自监督终身学习。该设定因数据非独立同分布性、单次遍历特性、缺乏外部监督及无先验知识而极具挑战性。为应对这些挑战，我们提出无先验知识的自监督对比终身学习算法（SCALE），该算法能纯粹从连续数据流中实时提取并记忆表征。SCALE围绕三个核心组件设计：伪监督对比损失、自监督遗忘损失、以及用于均匀子集选择的在线记忆更新机制。三个组件协同工作以最大化学习性能。我们在独立同分布及四种非独立同分布数据流下对SCALE进行全面实验，结果表明：在CIFAR-10、CIFAR-100和TinyImageNet数据集上，SCALE在所有设定下均超越当前最优算法，kNN准确率分别提升达3.83%、2.77%和5.86%。