The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications. Existing large-scale 3D perception datasets often contain data that are meticulously cleaned. Such configurations, however, cannot reflect the reliability of perception models during the deployment stage. In this work, we present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios against natural corruptions that occur in real-world environments. Specifically, we consider eight corruption types stemming from severe weather conditions, external disturbances, and internal sensor failure. We uncover that, although promising results have been progressively achieved on standard benchmarks, state-of-the-art 3D perception models are at risk of being vulnerable to corruptions. We draw key observations on the use of data representations, augmentation schemes, and training strategies, that could severely affect the model's performance. To pursue better robustness, we propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency. We hope our benchmark and approach could inspire future research in designing more robust and reliable 3D perception models. Our robustness benchmark suite is publicly available.

翻译：自然环境和传感器引起的干扰对3D感知系统的鲁棒性至关重要，尤其是在安全关键应用中。现有大规模3D感知数据集通常包含经过精细清洗的数据，然而这种配置无法反映感知模型在部署阶段的可靠性。本文提出Robo3D，这是首个系统评估3D检测器与分割器在真实环境自然干扰下分布外场景鲁棒性的综合基准。具体而言，我们考虑了由恶劣天气条件、外部扰动和内部传感器故障引发的八种干扰类型。研究表明，尽管标准基准测试上已逐步取得显著成果，但现有最优3D感知模型仍面临易受干扰影响的风险。我们揭示了数据表征、数据增强方案和训练策略的关键特性——这些因素会显著影响模型性能。为追求更优的鲁棒性，我们提出密度不敏感训练框架及简单灵活的体素化策略以增强模型韧性。期望本基准与方法能推动未来更鲁棒、更可靠3D感知模型的研究设计。相关鲁棒性基准套件现已开源。