Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models. This study introduces Calib3D, a pioneering effort to benchmark and scrutinize the reliability of 3D scene understanding models from an uncertainty estimation viewpoint. We comprehensively evaluate 28 state-of-the-art models across 10 diverse 3D datasets, uncovering insightful phenomena that cope with both the aleatoric and epistemic uncertainties in 3D scene understanding. We discover that despite achieving impressive levels of accuracy, existing models frequently fail to provide reliable uncertainty estimates -- a pitfall that critically undermines their applicability in safety-sensitive contexts. Through extensive analysis of key factors such as network capacity, LiDAR representations, rasterization resolutions, and 3D data augmentation techniques, we correlate these aspects directly with the model calibration efficacy. Furthermore, we introduce DeptS, a novel depth-aware scaling approach aimed at enhancing 3D model calibration. Extensive experiments across a wide range of configurations validate the superiority of our method. We hope this work could serve as a cornerstone for fostering reliable 3D scene understanding. Code and benchmark toolkits are publicly available.

翻译：安全关键的3D场景理解任务不仅要求3D感知模型具有准确的预测能力，更需要其具备可靠的置信度输出。本研究提出Calib3D，从不确定性估计角度系统评估和审视3D场景理解模型可靠性的开创性工作。我们对涵盖10个多样化3D数据集的28个最先进模型进行全面评估，揭示了应对3D场景理解中偶然不确定性与认知不确定性的深刻现象。研究发现，尽管现有模型已取得令人瞩目的精度水平，但其在提供可靠不确定性估计方面仍存在明显缺陷——这一关键问题严重制约了它们在安全敏感场景中的适用性。通过对网络容量、LiDAR表征、栅格化分辨率以及3D数据增强等关键因素的深入分析，我们直接将这些要素与模型校准效能建立关联。此外，我们提出DeptS——一种面向深度感知的尺度增强方法，旨在提升3D模型校准性能。在多种配置下的广泛实验验证了本方法的优越性。期望此工作能为推动可靠3D场景理解奠定基石。代码与基准工具包已公开提供。