For cyber-physical systems (CPS), including robotics and autonomous vehicles, mass deployment has been hindered by fatal errors that occur when operating in rare events. To replicate rare events such as vehicle crashes, many companies have created logging systems and employed crash reconstruction experts to meticulously recreate these valuable events in simulation. However, in these methods, "what if" questions are not easily formulated and answered. We present ScenarioNL, an AI System for creating scenario programs from natural language. Specifically, we generate these programs from police crash reports. Reports normally contain uncertainty about the exact details of the incidents which we represent through a Probabilistic Programming Language (PPL), Scenic. By using Scenic, we can clearly and concisely represent uncertainty and variation over CPS behaviors, properties, and interactions. We demonstrate how commonplace prompting techniques with the best Large Language Models (LLM) are incapable of reasoning about probabilistic scenario programs and generating code for low-resource languages such as Scenic. Our system is comprised of several LLMs chained together with several kinds of prompting strategies, a compiler, and a simulator. We evaluate our system on publicly available autonomous vehicle crash reports in California from the last five years and share insights into how we generate code that is both semantically meaningful and syntactically correct.

翻译：对于包括机器人和自动驾驶车辆在内的信息物理系统（CPS），大规模部署一直受到在罕见事件中运行时发生的致命错误的阻碍。为了复现车辆碰撞等罕见事件，许多公司建立了日志系统，并聘请事故重建专家在仿真中精心重现这些宝贵事件。然而，在这些方法中，"假设性"问题难以被提出和解答。我们提出了ScenarioNL，一种从自然语言创建场景程序的人工智能系统。具体而言，我们从警方事故报告中生成这些程序。报告通常包含对事件具体细节的不确定性，我们通过概率编程语言（PPL）Scenic来表征这种不确定性。通过使用Scenic，我们能够清晰简洁地表征CPS行为、属性及交互中的不确定性和变化。我们展示了使用最佳大语言模型（LLM）的常规提示技术无法对概率场景程序进行推理，也无法为Scenic等低资源语言生成代码。我们的系统由多个通过多种提示策略链接的LLM、一个编译器和一个模拟器组成。我们在过去五年加利福尼亚州公开可用的自动驾驶车辆事故报告上评估了我们的系统，并分享了如何生成既语义有意义又语法正确的代码的见解。