Simulations are gaining increasingly significance in the field of autonomous driving due to the demand for rapid prototyping and extensive testing. Employing physics-based simulation brings several benefits at an affordable cost, while mitigating potential risks to prototypes, drivers, and vulnerable road users. However, there exit two primary limitations. Firstly, the reality gap which refers to the disparity between reality and simulation and prevents the simulated autonomous driving systems from having the same performance in the real world. Secondly, the lack of empirical understanding regarding the behavior of real agents, such as backup drivers or passengers, as well as other road users such as vehicles, pedestrians, or cyclists. Agent simulation is commonly implemented through deterministic or randomized probabilistic pre-programmed models, or generated from real-world data; but it fails to accurately represent the behaviors adopted by real agents while interacting within a specific simulated scenario. This paper extends the description of our proposed framework to enable real-time interaction between real agents and simulated environments, by means immersive virtual reality and human motion capture systems within the CARLA simulator for autonomous driving. We have designed a set of usability examples that allow the analysis of the interactions between real pedestrians and simulated autonomous vehicles and we provide a first measure of the user's sensation of presence in the virtual environment.

翻译：模拟技术在自动驾驶领域的重要性日益凸显，这源于对快速原型开发和广泛测试的需求。采用基于物理的仿真能以可承受的成本带来诸多益处，同时降低对原型车、驾驶员及弱势道路使用者的潜在风险。然而，现有技术存在两大局限：其一是现实差距，即真实世界与模拟环境间的差异，这阻碍了自动驾驶系统在仿真环境中获得与真实世界等同的性能表现；其二是对后备驾驶员、乘客等真实智能体，以及车辆、行人、自行车等其他道路使用者行为缺乏实证性理解。当前智能体仿真通常通过确定性或随机概率预编程模型实现，或基于真实世界数据生成，但无法准确呈现真实智能体在特定仿真场景中交互时表现出的行为特征。本文深化了我们提出的框架描述，旨在通过沉浸式虚拟现实与人体运动捕捉系统，在CARLA自动驾驶仿真器中实现真实智能体与模拟环境的实时交互。我们设计了一套可用性示例，允许分析真实行人与仿真自动驾驶车辆之间的交互，并首次提供了用户在虚拟环境中临场感的量化测量指标。