For cyber-physical systems (CPS), including robotics and autonomous vehicles, mass deployment has been hindered by fatal errors that occur when operating in rare events. To replicate rare events such as vehicle crashes, many companies have created logging systems and employed crash reconstruction experts to meticulously recreate these valuable events in simulation. However, in these methods, "what if" questions are not easily formulated and answered. We present ScenarioNL, an AI System for creating scenario programs from natural language. Specifically, we generate these programs from police crash reports. Reports normally contain uncertainty about the exact details of the incidents which we represent through a Probabilistic Programming Language (PPL), Scenic. By using Scenic, we can clearly and concisely represent uncertainty and variation over CPS behaviors, properties, and interactions. We demonstrate how commonplace prompting techniques with the best Large Language Models (LLM) are incapable of reasoning about probabilistic scenario programs and generating code for low-resource languages such as Scenic. Our system is comprised of several LLMs chained together with several kinds of prompting strategies, a compiler, and a simulator. We evaluate our system on publicly available autonomous vehicle crash reports in California from the last five years and share insights into how we generate code that is both semantically meaningful and syntactically correct.

翻译：对于包括机器人和自动驾驶车辆在内的信息物理系统（CPS），大规模部署因罕见事件运行中发生的致命错误而受阻。为复现车辆碰撞等罕见事件，许多公司已构建日志系统并聘请碰撞重建专家，在仿真中细致重现这些宝贵事件。然而，在这些方法中，“假设”问题难以被轻松构建和解答。我们提出ScenarioNL——一个从自然语言创建场景程序的人工智能系统。具体而言，我们通过警方碰撞报告生成这些程序。报告通常包含对事件确切细节的不确定性，我们通过概率性编程语言（PPL）Scenic对其进行表示。借助Scenic，我们能够清晰简洁地表达CPS行为、属性和交互中的不确定性与变化。我们证明，使用最佳大语言模型（LLM）的常规提示技术无法推理概率性场景程序，也无法为Scenic等低资源语言生成代码。我们的系统由多个LLM通过多种提示策略、一个编译器和一个仿真器串联而成。我们基于加州过去五年公开的自动驾驶车辆碰撞报告评估该系统，并分享关于如何生成语义有效且语法正确代码的见解。