In autonomous driving, the temporal stability of 3D object detection greatly impacts the driving safety. However, the detection stability cannot be accessed by existing metrics such as mAP and MOTA, and consequently is less explored by the community. To bridge this gap, this work proposes Stability Index (SI), a new metric that can comprehensively evaluate the stability of 3D detectors in terms of confidence, box localization, extent, and heading. By benchmarking state-of-the-art object detectors on the Waymo Open Dataset, SI reveals interesting properties of object stability that have not been previously discovered by other metrics. To help models improve their stability, we further introduce a general and effective training strategy, called Prediction Consistency Learning (PCL). PCL essentially encourages the prediction consistency of the same objects under different timestamps and augmentations, leading to enhanced detection stability. Furthermore, we examine the effectiveness of PCL with the widely-used CenterPoint, and achieve a remarkable SI of 86.00 for vehicle class, surpassing the baseline by 5.48. We hope our work could serve as a reliable baseline and draw the community's attention to this crucial issue in 3D object detection. Codes will be made publicly available.
翻译:在自动驾驶中,三维目标检测的时间稳定性对行车安全具有重大影响。然而,现有指标(如mAP和MOTA)无法评估检测稳定性,导致该问题在学界较少被探索。为弥补这一空白,本文提出了稳定性指数(SI),这是一种能够从置信度、边界框定位、尺寸和航向角四个方面综合评估三维检测器稳定性的新指标。通过在Waymo开放数据集上对先进目标检测器进行基准测试,SI揭示了其他指标未曾发现的目标稳定性特性。为帮助模型提升稳定性,我们进一步引入了一种通用且有效的训练策略,称为预测一致性学习(PCL)。PCL的核心思想是促进同一目标在不同时间戳和数据增强条件下的预测一致性,从而增强检测稳定性。此外,我们使用广泛采用的CenterPoint模型验证了PCL的有效性,在车辆类别上取得了86.00的显著SI值,较基线提升5.48。我们希望本研究能为该领域提供可靠的基准,并引起学界对三维目标检测中这一关键问题的关注。代码将公开发布。