The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications. Existing large-scale 3D perception datasets often contain data that are meticulously cleaned. Such configurations, however, cannot reflect the reliability of perception models during the deployment stage. In this work, we present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios against natural corruptions that occur in real-world environments. Specifically, we consider eight corruption types stemming from adversarial weather conditions, external disturbances, and internal sensor failure. We uncover that, although promising results have been progressively achieved on standard benchmarks, state-of-the-art 3D perception models are at risk of being vulnerable to corruptions. We draw key observations on the use of data representations, augmentation schemes, and training strategies, that could severely affect the model's performance. To pursue better robustness, we propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency. We hope our benchmark and approach could inspire future research in designing more robust and reliable 3D perception models. Our robustness benchmark suite is publicly available.

翻译：三维感知系统在环境和传感器自然扰动下的鲁棒性对安全关键型应用至关重要。现有大规模三维感知数据集往往包含经过精心清洗的数据，然而此类配置无法反映感知模型在部署阶段的可靠性。本文提出Robo3D——首个系统探究三维检测器与分割器在真实环境自然扰动引发的分布外场景中鲁棒性的综合基准。具体而言，我们考虑了源于恶劣天气条件、外部干扰及内部传感器故障的八类扰动。研究发现，尽管标准基准上已逐步取得显著成果，但最先进的三维感知模型仍面临易受扰动影响的风险。我们通过关键观察揭示了数据表示、数据增强方案及训练策略的选用会显著影响模型性能。为追求更优的鲁棒性，我们提出了密度不敏感训练框架，并辅以简单灵活的体素化策略来增强模型抗干扰能力。希望本基准与研发方法能启发未来更鲁棒可靠的三维感知模型设计。我们的鲁棒性基准套件现已公开。