We propose UnCLe, a standardized benchmark for Unsupervised Continual Learning of a multimodal depth estimation task: Depth completion aims to infer a dense depth map from a pair of synchronized RGB image and sparse depth map. We benchmark depth completion models under the practical scenario of unsupervised learning over continuous streams of data. Existing methods are typically trained on a static, or stationary, dataset. However, when adapting to novel non-stationary distributions, they "catastrophically forget" previously learned information. UnCLe simulates these non-stationary distributions by adapting depth completion models to sequences of datasets containing diverse scenes captured from distinct domains using different visual and range sensors. We adopt representative methods from continual learning paradigms and translate them to enable unsupervised continual learning of depth completion. We benchmark these models for indoor and outdoor and investigate the degree of catastrophic forgetting through standard quantitative metrics. Furthermore, we introduce model inversion quality as an additional measure of forgetting. We find that unsupervised continual learning of depth completion is an open problem, and we invite researchers to leverage UnCLe as a development platform.

翻译：我们提出了UnCLe，这是一个用于多模态深度估计任务无监督持续学习的标准化基准：深度补全旨在从同步的RGB图像和稀疏深度图对中推断出稠密深度图。我们在连续数据流上进行无监督学习的实际场景下对深度补全模型进行基准测试。现有方法通常在静态或平稳的数据集上进行训练。然而，当适应新的非平稳分布时，它们会"灾难性地遗忘"先前学习到的信息。UnCLe通过使深度补全模型适应包含不同场景序列的数据集来模拟这些非平稳分布，这些场景使用不同的视觉和距离传感器从不同领域捕获。我们采用持续学习范式的代表性方法，并将其转化为能够实现深度补全的无监督持续学习。我们对这些模型在室内和室外场景进行基准测试，并通过标准定量指标研究灾难性遗忘的程度。此外，我们引入模型反转质量作为遗忘的附加度量。我们发现深度补全的无监督持续学习是一个开放性问题，并邀请研究人员利用UnCLe作为开发平台。