Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments. Optimally, the robot is able to adapt by itself to new conditions without human supervision, e.g., automatically adjusting its perception system to changing lighting conditions. In this work, we address the task of continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We introduce CoDEPS to perform continual learning involving multiple real-world domains while mitigating catastrophic forgetting by leveraging experience replay. In particular, we propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. Furthermore, we explicitly address the limited storage capacity of robotic systems by leveraging sampling strategies for constructing a fixed-size replay buffer based on rare semantic class sampling and image diversity. We perform extensive evaluations of CoDEPS on various real-world datasets demonstrating that it successfully adapts to unseen environments without sacrificing performance on previous domains while achieving state-of-the-art results. The code of our work is publicly available at http://codeps.cs.uni-freiburg.de.

翻译：在开放世界环境中运行机器人需要其对未见过的场景具备高度鲁棒性。理想情况下，机器人能够在无需人类监督的条件下自主适应新环境，例如自动调整感知系统以适应光照条件变化。本文针对深度学习驱动的单目深度估计与全景分割任务，提出一种在新环境中进行在线持续学习的方法。我们引入CoDEPS框架，通过经验回放机制缓解灾难性遗忘，实现跨多个真实世界领域的持续学习。具体而言，我们提出一种新颖的域混合策略生成伪标签以适配全景分割任务。此外，我们通过基于稀有语义类别采样和图像多样性的采样策略构建固定容量的回放缓冲区，显式解决了机器人系统存储容量受限的问题。我们在多个真实数据集上对CoDEPS进行了全面评估，结果表明该方法在成功适应未知环境的同时未牺牲先前领域的性能，且达到了当前最优水平。本研究的代码已开源至http://codeps.cs.uni-freiburg.de。