For effective human-robot interaction, robots need to understand, plan, and execute complex, long-horizon tasks described by natural language. Recent advances in large language models (LLMs) have shown promise for translating natural language into robot action sequences for complex tasks. However, existing approaches either translate the natural language directly into robot trajectories or factor the inference process by decomposing language into task sub-goals and relying on a motion planner to execute each sub-goal. When complex environmental and temporal constraints are involved, inference over planning tasks must be performed jointly with motion plans using traditional task-and-motion planning (TAMP) algorithms, making factorization into subgoals untenable. Rather than using LLMs to directly plan task sub-goals, we instead perform few-shot translation from natural language task descriptions to an intermediate task representation that can then be consumed by a TAMP algorithm to jointly solve the task and motion plan. To improve translation, we automatically detect and correct both syntactic and semantic errors via autoregressive re-prompting, resulting in significant improvements in task completion. We show that our approach outperforms several methods using LLMs as planners in complex task domains. See our project website https://yongchao98.github.io/MIT-REALM-AutoTAMP/ for prompts, videos, and code.

翻译：摘要：为了实现有效的人机交互，机器人需要理解、规划并执行由自然语言描述的复杂、长期任务。大型语言模型（LLM）的最新进展已展现出将自然语言翻译为复杂机器人动作序列的潜力。然而，现有方法要么直接翻译自然语言为机器人轨迹，要么通过将语言分解为任务子目标并依赖运动规划器执行每个子目标来分解推理过程。当涉及复杂环境与时间约束时，规划任务的推理必须与运动规划联合执行，需借助传统任务与运动规划（TAMP）算法，这使得分解为子目标的方法难以奏效。我们不使用LLM直接规划任务子目标，而是基于少样本学习将自然语言任务描述翻译为可被TAMP算法消费的中间任务表示，从而联合求解任务与运动规划。为提升翻译质量，我们通过自回归重新提示自动检测并纠正语法和语义错误，显著提高了任务完成率。实验表明，在复杂任务领域中，我们的方法优于多种使用LLM作为规划器的方案。提示词、视频和代码请参见项目网站 https://yongchao98.github.io/MIT-REALM-AutoTAMP/。