Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments. Optimally, the robot is able to adapt by itself to new conditions without human supervision, e.g., automatically adjusting its perception system to changing lighting conditions. In this work, we address the task of continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We introduce CoDEPS to perform continual learning involving multiple real-world domains while mitigating catastrophic forgetting by leveraging experience replay. In particular, we propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. Furthermore, we explicitly address the limited storage capacity of robotic systems by proposing sampling strategies for constructing a fixed-size replay buffer based on rare semantic class sampling and image diversity. We perform extensive evaluations of CoDEPS on various real-world datasets demonstrating that it successfully adapts to unseen environments without sacrificing performance on previous domains while achieving state-of-the-art results. The code of our work is publicly available at http://codeps.cs.uni-freiburg.de.

翻译：在开放世界中运行的机器人需要具备对未见环境的高度鲁棒性。理想情况下，机器人能够在无需人工监督的条件下自主学习适应新环境，例如自动调整其感知系统以应对光照变化。本文针对基于深度学习的单目深度估计与全景分割在新环境中的持续学习问题，提出了一种在线式解决方案。我们引入了CoDEPS框架，通过经验回放机制缓解灾难性遗忘，实现在多个真实世界域上的持续学习。具体而言，我们提出了一种新颖的域混合策略来生成伪标签，从而适配全景分割任务。此外，我们针对机器人系统中存储容量有限的问题，基于稀有语义类别采样与图像多样性设计了固定大小回放缓冲区的构建策略。通过在多个真实世界数据集上的全面评估，我们证明了CoDEPS能够成功适应未见环境，且不牺牲在先前域上的性能，同时达到最优结果。本工作的代码已公开于http://codeps.cs.uni-freiburg.de。