Large Language Models (LLMs) have been shown to be capable of performing high-level planning for long-horizon robotics tasks, yet existing methods require access to a pre-defined skill library (e.g. picking, placing, pulling, pushing, navigating). However, LLM planning does not address how to design or learn those behaviors, which remains challenging particularly in long-horizon settings. Furthermore, for many tasks of interest, the robot needs to be able to adjust its behavior in a fine-grained manner, requiring the agent to be capable of modifying low-level control actions. Can we instead use the internet-scale knowledge from LLMs for high-level policies, guiding reinforcement learning (RL) policies to efficiently solve robotic control tasks online without requiring a pre-determined set of skills? In this paper, we propose Plan-Seq-Learn (PSL): a modular approach that uses motion planning to bridge the gap between abstract language and learned low-level control for solving long-horizon robotics tasks from scratch. We demonstrate that PSL achieves state-of-the-art results on over 25 challenging robotics tasks with up to 10 stages. PSL solves long-horizon tasks from raw visual input spanning four benchmarks at success rates of over 85%, out-performing language-based, classical, and end-to-end approaches. Video results and code at https://mihdalal.github.io/planseqlearn/

翻译：大型语言模型（LLMs）已被证明能够为长时域机器人任务执行高层规划，但现有方法需访问预定义技能库（如抓取、放置、拉拽、推动、导航）。然而，LLM规划并未解决如何设计或学习这些行为的问题——这一挑战在长时域场景中尤为突出。此外，对于许多目标任务，机器人需要以细粒度方式调整行为，要求智能体具备修改底层控制动作的能力。我们能否利用LLMs的互联网规模知识构建高层策略，引导强化学习（RL）策略在线高效解决机器人控制任务，而无需预定义技能集？本文提出Plan-Seq-Learn（PSL）：一种模块化方法，通过运动规划弥合抽象语言与学习型底层控制之间的鸿沟，从零解决长时域机器人任务。我们证明PSL在超过25项包含多达10个阶段的具有挑战性的机器人任务上取得了最先进的结果。PSL从原始视觉输入出发，在四个基准测试中以超过85%的成功率解决长时域任务，性能优于基于语言的方法、经典方法和端到端方法。视频结果和代码见https://mihdalal.github.io/planseqlearn/