Autonomous Driving System (ADS) testing is crucial in ADS development, with the current primary focus being on safety. However, the evaluation of non-safety-critical performance, particularly the ADS's ability to make optimal decisions and produce optimal paths for autonomous vehicles (AVs), is also vital to ensure the intelligence and reduce risks of AVs. Currently, there is little work dedicated to assessing the robustness of ADSs' path-planning decisions (PPDs), i.e., whether an ADS can maintain the optimal PPD after an insignificant change in the environment. The key challenges include the lack of clear oracles for assessing PPD optimality and the difficulty in searching for scenarios that lead to non-optimal PPDs. To fill this gap, in this paper, we focus on evaluating the robustness of ADSs' PPDs and propose the first method, Decictor, for generating non-optimal decision scenarios (NoDSs), where the ADS does not plan optimal paths for AVs. Decictor comprises three main components: Non-invasive Mutation, Consistency Check, and Feedback. To overcome the oracle challenge, Non-invasive Mutation is devised to implement conservative modifications, ensuring the preservation of the original optimal path in the mutated scenarios. Subsequently, the Consistency Check is applied to determine the presence of non-optimal PPDs by comparing the driving paths in the original and mutated scenarios. To deal with the challenge of large environment space, we design Feedback metrics that integrate spatial and temporal dimensions of the AV's movement. These metrics are crucial for effectively steering the generation of NoDSs. We evaluate Decictor on Baidu Apollo, an open-source and production-grade ADS. The experimental results validate the effectiveness of Decictor in detecting non-optimal PPDs of ADSs.

翻译：自动驾驶系统（ADS）测试在ADS开发中至关重要，当前主要关注点在于安全性。然而，非安全关键性能的评估，特别是ADS为自动驾驶车辆（AV）做出最优决策并生成最优路径的能力，对于确保AV的智能性并降低其风险同样至关重要。目前，鲜有工作致力于评估ADS路径规划决策（PPD）的鲁棒性，即ADS能否在环境发生微小变化后仍保持最优PPD。关键挑战包括缺乏评估PPD最优性的明确预言机制，以及难以搜索导致非最优PPD的场景。为填补这一空白，本文聚焦于评估ADS的PPD鲁棒性，并提出了首个用于生成非最优决策场景（NoDS）的方法Decictor，在该类场景中ADS无法为AV规划最优路径。Decictor包含三个核心组件：非侵入式变异、一致性检查和反馈机制。为克服预言机制缺失的挑战，非侵入式变异被设计用于实施保守修改，确保变异场景中原始最优路径得以保留。随后，通过比较原始场景与变异场景中的行驶路径，应用一致性检查来判断是否存在非最优PPD。为应对庞大环境空间的挑战，我们设计了融合AV运动时空维度的反馈度量指标。这些指标对于有效引导NoDS的生成至关重要。我们在开源且达到生产级的自动驾驶系统百度Apollo上评估了Decictor。实验结果验证了Decictor在检测ADS非最优PPD方面的有效性。