For cyber-physical systems (CPS), including robotics and autonomous vehicles, mass deployment has been hindered by fatal errors that occur when operating in rare events. To replicate rare events such as vehicle crashes, many companies have created logging systems and employed crash reconstruction experts to meticulously recreate these valuable events in simulation. However, in these methods, "what if" questions are not easily formulated and answered. We present ScenarioNL, an AI System for creating scenario programs from natural language. Specifically, we generate these programs from police crash reports. Reports normally contain uncertainty about the exact details of the incidents which we represent through a Probabilistic Programming Language (PPL), Scenic. By using Scenic, we can clearly and concisely represent uncertainty and variation over CPS behaviors, properties, and interactions. We demonstrate how commonplace prompting techniques with the best Large Language Models (LLM) are incapable of reasoning about probabilistic scenario programs and generating code for low-resource languages such as Scenic. Our system is comprised of several LLMs chained together with several kinds of prompting strategies, a compiler, and a simulator. We evaluate our system on publicly available autonomous vehicle crash reports in California from the last five years and share insights into how we generate code that is both semantically meaningful and syntactically correct.

翻译：针对包含机器人和自主车辆在内的信息物理系统（CPS），因罕见事件运行中出现的致命错误阻碍了其大规模部署。为复现车辆碰撞等罕见事件，众多企业已建立日志系统并聘请碰撞重建专家，在仿真环境中精细重建这些关键事件。然而，此类方法难以有效构建并解答"假设性"问题。我们提出ScenarioNL这一人工智能系统，通过自然语言创建场景程序，具体从警方事故报告中生成程序。事故报告通常包含事件细节的不确定性，我们通过概率编程语言（PPL）Scenic对这种不确定性进行表征。借助Scenic，可清晰简洁地表达CPS行为、属性及交互中的不确定性与变异性。我们论证了常规提示策略与最优大语言模型（LLM）无法有效推理概率场景程序，亦难以生成Scenic等低资源语言代码。我们的系统由多个LLM经多类提示策略串联而成，并集成编译器与仿真器。基于加州近五年公开的自主车辆碰撞报告对系统进行评估，并分享生成兼具语义合理性与语法正确性代码的实践洞察。