With increasing complexity of Automated Driving Systems (ADS), ensuring their safety and reliability has become a critical challenge. The Verification and Validation (V&V) of these systems are particularly demanding when AI components are employed to implement perception and/or control functions. In ESA-funded project VIVAS, we developed a generic framework for system-level simulation-based V&V of autonomous systems. The approach is based on a simulation model of the system, an abstract model that describes symbolically the system behavior, and formal methods to generate scenarios and verify the simulation executions. Various coverage criteria can be defined to guide the automated generation of the scenarios. In this paper, we describe the instantiation of the VIVAS framework for an ADS case study. This is based on the integration of CARLA, a widely-used driving simulator, and its ScenarioRunner tool, which enables the creation of diverse and complex driving scenarios. This is also used in the CARLA Autonomous Driving Challenge to validate different ADS agents for perception and control based on AI, shared by the CARLA community. We describe the development of an abstract ADS model and the formulation of a coverage criterion that focuses on the behaviors of vehicles relative to the vehicle with ADS under verification. Leveraging the VIVAS framework, we generate and execute various driving scenarios, thus testing the capabilities of the AI components. The results show the effectiveness of VIVAS in automatically generating scenarios for system-level simulation-based V&V of an automated driving system using CARLA and ScenarioRunner. Therefore, they highlight the potential of the approach as a powerful tool in the future of ADS V&V methodologies.

翻译：随着自动驾驶系统（ADS）复杂性的不断增加，确保其安全性和可靠性已成为关键挑战。当采用人工智能组件实现感知和/或控制功能时，这些系统的验证与确认（V&V）工作尤为艰巨。在欧空局资助的VIVAS项目中，我们开发了一个用于自主系统系统级仿真验证与确认的通用框架。该方法基于系统的仿真模型、符号化描述系统行为的抽象模型，以及用于生成场景和验证仿真执行的正式方法。可定义多种覆盖率指标以指导场景的自动生成。本文描述了VIVAS框架在自动驾驶系统案例研究中的实例化过程。该实例化基于广泛使用的驾驶仿真器CARLA及其场景生成工具ScenarioRunner的集成，后者支持创建多样化的复杂驾驶场景。该框架也被用于CARLA自动驾驶挑战赛，以验证CARLA社区共享的基于AI的感知与控制的多种ADS智能体。我们描述了抽象ADS模型的开发过程，以及聚焦于待验证ADS车辆与其他车辆相对行为的覆盖率指标制定。借助VIVAS框架，我们生成并执行了多种驾驶场景，从而测试了AI组件的性能。结果表明，VIVAS在使用CARLA和ScenarioRunner进行自动驾驶系统系统级仿真验证与确认时，能够有效自动生成场景。因此，这些成果凸显了该方法作为未来ADS验证与确认方法论中强大工具的潜力。