Self-driving vehicles (SDVs) must be rigorously tested on a wide range of scenarios to ensure safe deployment. The industry typically relies on closed-loop simulation to evaluate how the SDV interacts on a corpus of synthetic and real scenarios and verify it performs properly. However, they primarily only test the system's motion planning module, and only consider behavior variations. It is key to evaluate the full autonomy system in closed-loop, and to understand how variations in sensor data based on scene appearance, such as the shape of actors, affect system performance. In this paper, we propose a framework, Adv3D, that takes real world scenarios and performs closed-loop sensor simulation to evaluate autonomy performance, and finds vehicle shapes that make the scenario more challenging, resulting in autonomy failures and uncomfortable SDV maneuvers. Unlike prior works that add contrived adversarial shapes to vehicle roof-tops or roadside to harm perception only, we optimize a low-dimensional shape representation to modify the vehicle shape itself in a realistic manner to degrade autonomy performance (e.g., perception, prediction, and motion planning). Moreover, we find that the shape variations found with Adv3D optimized in closed-loop are much more effective than those in open-loop, demonstrating the importance of finding scene appearance variations that affect autonomy in the interactive setting.

翻译：自动驾驶车辆必须在广泛场景中经过严格测试以确保安全部署。工业界通常依赖闭环仿真来评估自动驾驶车辆与合成及真实场景库的交互，并验证其性能是否正常。然而，现有方法主要测试系统的运动规划模块，且仅考虑行为变化。关键问题在于，需在闭环条件下评估完整自主系统，并理解基于场景外观的传感器数据变化（如道路参与者形状）如何影响系统性能。本文提出AdV3D框架，该框架利用真实世界场景执行闭环传感器仿真以评估自主性能，并发现使场景更具挑战性的车辆形状，从而导致自主系统故障和自动驾驶车辆的不舒适操作。不同于以往在车顶或路边添加刻意对抗形状以仅破坏感知模块的研究，我们优化低维形状表示，以逼真的方式修改车辆形状本身，从而降低自主系统性能（包括感知、预测和运动规划）。此外，我们发现经闭环优化的AdV3D形状变化比开环优化更有效，这证明在交互式场景中寻找影响自主系统性能的外观变化至关重要。