Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models. This study introduces Calib3D, a pioneering effort to benchmark and scrutinize the reliability of 3D scene understanding models from an uncertainty estimation viewpoint. We comprehensively evaluate 28 state-of-the-art models across 10 diverse 3D datasets, uncovering insightful phenomena that cope with both the aleatoric and epistemic uncertainties in 3D scene understanding. We discover that despite achieving impressive levels of accuracy, existing models frequently fail to provide reliable uncertainty estimates -- a pitfall that critically undermines their applicability in safety-sensitive contexts. Through extensive analysis of key factors such as network capacity, LiDAR representations, rasterization resolutions, and 3D data augmentation techniques, we correlate these aspects directly with the model calibration efficacy. Furthermore, we introduce DeptS, a novel depth-aware scaling approach aimed at enhancing 3D model calibration. Extensive experiments across a wide range of configurations validate the superiority of our method. We hope this work could serve as a cornerstone for fostering reliable 3D scene understanding. Code and benchmark toolkit are publicly available.

翻译：安全关键的三维场景理解任务不仅要求三维感知模型提供准确的预测，还需要其具备可靠的置信度。本研究提出了Calib3D，首次从不确定性估计的视角对三维场景理解模型的可靠性进行系统性基准测试与深入分析。我们在10个多样化的三维数据集上全面评估了28个前沿模型，揭示了与三维场景理解中偶然不确定性和认知不确定性相关的深刻现象。我们发现，尽管现有模型在精度上取得了显著成就，却常常无法提供可靠的不确定性估计——这一缺陷严重削弱了其在安全敏感场景中的适用性。通过对网络容量、激光雷达表示形式、栅格化分辨率及三维数据增强技术等关键因素进行广泛分析，我们直接将这些方面与模型校准效能关联起来。此外，我们提出了DeptS，一种新颖的深度感知缩放方法，旨在提升三维模型的校准能力。在多种配置下的大量实验验证了我们方法的优越性。我们希望这项工作能够成为推动可靠三维场景理解发展的基石。代码与基准测试工具包已公开提供。