Autonomous Vehicles (AVs) have the potential to provide numerous societal benefits, such as decreased road accidents and increased overall transportation efficiency. However, quantifying the risk associated with AVs is challenging due to the lack of historical data and the rapidly evolving technology. This paper presents a data-driven framework for comparing the risk of different AVs' behaviors in various operational design domains (ODDs), based on counterfactual simulations of "misbehaving" road users. We introduce the concept of counterfactual safety margin, which represents the minimum deviation from normal behavior that could lead to a collision. This concept helps to find the most critical scenarios but also to assess the frequency and severity of risk of AVs. We show that the proposed methodology is applicable even when the AV's behavioral policy is unknown -- through worst- and best-case analyses -- making the method useful also to external third-party risk assessors. Our experimental results demonstrate the correlation between the safety margin, the driving policy quality, and the ODD shedding light on the relative risk associated with different AV providers. This work contributes to AV safety assessment and aids in addressing legislative and insurance concerns surrounding this emerging technology.

翻译：自主驾驶车辆（AVs）有望带来诸多社会效益，例如减少道路事故并提升整体交通效率。然而，由于缺乏历史数据且技术快速演进，量化AVs相关风险颇具挑战性。本文基于对"行为失常"道路使用者的反事实仿真，提出了一种数据驱动框架，用于比较不同AVs在各类运行设计域（ODDs）中的行为风险。我们引入反事实安全裕度的概念，该概念表示可能引发碰撞的最小正常行为偏差。这一概念不仅有助于发现最关键的场景，还能评估AVs风险的频率与严重程度。我们证明，即使AVs的行为策略未知（通过最坏与最佳情况分析），所提出的方法依然适用，从而使其对外部第三方风险评估者也具有实用性。实验结果展示了安全裕度、驾驶策略质量以及ODD之间的相关性，揭示了不同AV供应商的相对风险。本研究有助于AVs安全性评估，并协助解决围绕这一新兴技术的立法与保险相关问题。