Deploying 3D detectors in unfamiliar domains has been demonstrated to result in a drastic drop of up to 70-90% in detection rate due to variations in lidar, geographical region, or weather conditions from their original training dataset. This domain gap leads to missing detections for densely observed objects, misaligned confidence scores, and increased high-confidence false positives, rendering the detector highly unreliable. To address this, we introduce MS3D++, a self-training framework for multi-source unsupervised domain adaptation in 3D object detection. MS3D++ provides a straightforward approach to domain adaptation by generating high-quality pseudo-labels, enabling the adaptation of 3D detectors to a diverse range of lidar types, regardless of their density. Our approach effectively fuses predictions of an ensemble of multi-frame pre-trained detectors from different source domains to improve domain generalization. We subsequently refine the predictions temporally to ensure temporal consistency in box localization and object classification. Furthermore, we present an in-depth study into the performance and idiosyncrasies of various 3D detector components in a cross-domain context, providing valuable insights for improved cross-domain detector ensembling. Experimental results on Waymo, nuScenes and Lyft demonstrate that detectors trained with MS3D++ pseudo-labels achieve state-of-the-art performance, comparable to training with human-annotated labels in Bird's Eye View (BEV) evaluation for both low and high density lidar.
翻译:将3D检测器部署于陌生领域时,由于激光雷达类型、地理区域或天气条件与原始训练数据集的差异,检测率会急剧下降高达70-90%。这种领域差距导致密集观测目标的漏检、置信度分数错位以及高置信度假正例增多,使检测器极不可靠。为解决该问题,我们提出MS3D++——一个用于3D目标检测中多源无监督领域自适应的自训练框架。MS3D++通过生成高质量伪标签提供了一种直接的领域自适应方法,使3D检测器能够适应各种密度类型的激光雷达。我们的方法有效融合来自不同源域的多帧预训练检测器集成预测结果,以提升领域泛化能力。随后我们对预测结果进行时序精化,确保边界框定位与目标分类的时序一致性。进一步地,我们深入研究了跨域场景下各类3D检测器组件的性能特性与特殊表现,为改进跨域检测器集成提供了宝贵见解。在Waymo、nuScenes和Lyft上的实验表明:使用MS3D++伪标签训练的检测器在鸟瞰图(BEV)评估中,无论是低密度还是高密度激光雷达场景,均达到了与人工标注标签训练相媲美的先进性能水平。