Proprietary Autonomous Driving Systems are typically evaluated through disengagements, unplanned manual interventions to alter vehicle behavior, as annually reported by the California Department of Motor Vehicles. However, the real-world capabilities of prototypical open-source Level 4 vehicles over substantial distances remain largely unexplored. This study evaluates a research vehicle running an Autoware-based software stack across 236 km of mixed traffic. By classifying 30 disengagements across 26 rides with a novel five-level criticality framework, we observed a spatial disengagement rate of 0.127 1/km. Interventions predominantly occurred at lower speeds near static objects and traffic lights. Perception and Planning failures accounted for 40% and 26.7% of disengagements, respectively, largely due to object-tracking losses and operational deadlocks caused by parked vehicles. Frequent, unnecessary interventions highlighted a lack of trust on the part of the safety driver. These results show that while open-source software enables extensive operations, disengagement analysis is vital for uncovering robustness issues missed by standard metrics.

翻译：专有自动驾驶系统通常通过脱离（即加州机动车辆管理局年度报告中所述的、为改变车辆行为而进行的非计划性人工干预）进行评估。然而，原型开源L4级车辆在长距离行驶中的真实能力仍鲜有探究。本研究对搭载基于Autoware软件栈的研究车辆在236公里混合交通场景中的表现进行了评估。通过采用新型五级关键性框架对26次行程中的30次脱离事件进行分类，我们观察到空间脱离率为0.127 1/公里。干预主要发生在低速行驶状态下，且多邻近静态物体和交通信号灯。感知与规划模块故障分别占脱离事件的40%和26.7%，主要归因于目标跟踪丢失及由停放车辆引发的操作死锁。频繁的非必要干预暴露出安全驾驶员对系统的信任不足。研究结果表明，尽管开源软件能够支撑大规模运行，但脱离分析对于揭示标准指标无法捕捉的鲁棒性缺陷至关重要。