In autonomous driving, environment perception has significantly advanced with the utilization of deep learning techniques for diverse sensors such as cameras, depth sensors, or infrared sensors. The diversity in the sensor stack increases the safety and contributes to robustness against adverse weather and lighting conditions. However, the variance in data acquired from different sensors poses challenges. In the context of continual learning (CL), incremental learning is especially challenging for considerably large domain shifts, e.g. different sensor modalities. This amplifies the problem of catastrophic forgetting. To address this issue, we formulate the concept of modality-incremental learning and examine its necessity, by contrasting it with existing incremental learning paradigms. We propose the use of a modified Relevance Mapping Network (RMN) to incrementally learn new modalities while preserving performance on previously learned modalities, in which relevance maps are disjoint. Experimental results demonstrate that the prevention of shared connections in this approach helps alleviate the problem of forgetting within the constraints of a strict continual learning framework.

翻译：在自动驾驶领域，通过利用深度学习技术处理摄像头、深度传感器或红外传感器等多种传感器数据，环境感知能力得到了显著提升。传感器组合的多样性增强了安全性，并提高了对恶劣天气和光照条件的鲁棒性。然而，不同传感器获取数据的差异性带来了挑战。在持续学习（CL）背景下，对于存在显著领域偏移（例如不同传感器模态）的情况，增量学习尤为困难，这加剧了灾难性遗忘问题。为解决该问题，我们通过对比现有增量学习范式，提出了模态增量学习的概念并论证其必要性。我们提出采用改进的关联映射网络（RMN）来增量学习新模态，同时保持对已学习模态的性能，其中关联映射是相互分离的。实验结果表明，在严格的持续学习框架约束下，该方法通过避免共享连接有助于缓解遗忘问题。