Today's classical planners are powerful, but modeling input tasks in formats such as PDDL is tedious and error-prone. In contrast, planning with Large Language Models (LLMs) allows for almost any input text, but offers no guarantees on plan quality or even soundness. In an attempt to merge the best of these two approaches, some work has begun to use LLMs to automate parts of the PDDL creation process. However, these methods still require various degrees of expert input. We present NL2Plan, the first domain-agnostic offline LLM-driven planning system. NL2Plan uses an LLM to incrementally extract the necessary information from a short text prompt before creating a complete PDDL description of both the domain and the problem, which is finally solved by a classical planner. We evaluate NL2Plan on four planning domains and find that it solves 10 out of 15 tasks - a clear improvement over a plain chain-of-thought reasoning LLM approach, which only solves 2 tasks. Moreover, in two out of the five failure cases, instead of returning an invalid plan, NL2Plan reports that it failed to solve the task. In addition to using NL2Plan in end-to-end mode, users can inspect and correct all of its intermediate results, such as the PDDL representation, increasing explainability and making it an assistive tool for PDDL creation.
翻译:当今的经典规划器功能强大,但以PDDL等格式建模输入任务既繁琐又易出错。相比之下,使用大语言模型(LLM)进行规划几乎允许任何输入文本,但无法保证规划质量甚至正确性。为了融合这两种方法的优势,已有研究开始利用LLM自动化PDDL创建过程的某些部分,但这些方法仍需不同程度的专家输入。本文提出NL2Plan,首个领域无关的离线LLM驱动规划系统。NL2Plan通过LLM从简短文本提示中逐步提取必要信息,继而生成完整的领域与问题PDDL描述,最终由经典规划器求解。我们在四个规划领域上评估NL2Plan,发现其能解决15个任务中的10个——这明显优于仅解决2个任务的普通思维链推理LLM方法。此外,在五个失败案例中,有两个案例NL2Plan未返回无效规划,而是报告无法求解。除端到端使用外,用户可检查并修正其所有中间结果(如PDDL表示),这不仅提升了可解释性,还使其成为PDDL创建的辅助工具。