Large Language Models (LLMs) are capable of transforming natural language domain descriptions into plausibly looking PDDL markup. However, ensuring that actions are consistent within domains still remains a challenging task. In this paper we present a novel concept to significantly improve the quality of LLM-generated PDDL models by performing automated consistency checking during the generation process. Although the proposed consistency checking strategies still can't guarantee absolute correctness of generated models, they can serve as valuable source of feedback reducing the amount of correction efforts expected from a human in the loop. We demonstrate the capabilities of our error detection approach on a number of classical and custom planning domains (logistics, gripper, tyreworld, household, pizza).
翻译:大型语言模型(LLMs)能够将自然语言领域描述转换为看似合理的PDDL标记。然而,确保领域内动作的一致性仍是一项具有挑战性的任务。本文提出了一种新颖的概念,通过在生成过程中执行自动一致性检查,显著提升LLM生成的PDDL模型的质量。尽管所提出的一致性检查策略仍无法保证生成模型的绝对正确性,但它们可作为有价值的反馈来源,减少对人工参与循环中修正工作的预期。我们在多个经典和自定义规划领域(物流、抓取器、轮胎世界、家居、比萨)上展示了错误检测方法的能力。