Autonomous driving systems (ADSs) rely on real-time sensor data, such as cameras and LiDARs, for time-critical decisions using deep neural networks. The accuracy of these decisions is crucial for the widespread adoption of ADSs, as errors can have serious consequences. 3D obstacle detection, in particular, is sensitive to point cloud data (PCD) noise from various sources. However, the robustness of current 3D obstacle detection models against specification-based perturbations remains unevaluated. These perturbations are derived from the specification of LiDAR sensors and previous research on LiDAR's ability to capture objects of different colors and materials. They can manifest as very subtle sensor-based noises or obstacle-specific perturbations. Hence, we propose SORBET, a framework that tests the robustness of 3D obstacle detection models in ADS against such perturbations to the PCD to evaluate their robustness. We applied SORBET to evaluate the robustness of five classic 3D obstacle detection models, including one from an industry-grade Level 4 ADS (Baidu's Apollo). Furthermore, we studied how the deviated obstacle detection results would propagate and negatively impact trajectory prediction. Our evaluation emphasizes the importance of testing 3D obstacle detection against specification-based perturbations. We find that even very subtle changes in the PCD (i.e., removing two points) may introduce a non-trivial decrease in the detection performance. Furthermore, such a negative impact will further propagate to other modules and endanger the safety of the ADS.

翻译：自动驾驶系统依赖摄像头和激光雷达等实时传感器数据，通过深度神经网络进行时间关键型决策。这些决策的准确性对自动驾驶系统的广泛应用至关重要，因为错误可能导致严重后果。三维障碍物检测尤其容易受到来自不同源的点云数据噪声的影响。然而，当前三维障碍物检测模型对基于规范扰动的鲁棒性尚未得到充分评估。这些扰动源自激光雷达传感器规范以及先前关于激光雷达捕获不同颜色和材质物体能力的研究，可能表现为极其细微的基于传感器的噪声或特定于障碍物的扰动。为此，我们提出SORBET框架，通过向点云数据施加此类扰动来测试自动驾驶系统中三维障碍物检测模型的鲁棒性。我们应用SORBET评估了五种经典三维障碍物检测模型的鲁棒性，其中包括一个工业级L4自动驾驶系统（百度Apollo）的模型。此外，我们研究了偏差的障碍物检测结果如何传播并对轨迹预测产生负面影响。我们的评估强调了针对基于规范扰动测试三维障碍物检测的重要性。研究发现，即使点云数据发生极细微的变化（例如移除两个点），也可能导致检测性能显著下降。这种负面影响将进一步传播至其他模块，危及自动驾驶系统的安全性。