Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models. Unfortunately, applying such models to settings with embodied agents, such as robots, is challenging due to their lack of experience with the physical world, inability to parse non-language observations, and ignorance of rewards or safety constraints that robots may require. On the other hand, language-conditioned robotic policies that learn from interaction data can provide the necessary grounding that allows the agent to be correctly situated in the real world, but such policies are limited by the lack of high-level semantic understanding due to the limited breadth of the interaction data available for training them. Thus, if we want to make use of the semantic knowledge in a language model while still situating it in an embodied setting, we must construct an action sequence that is both likely according to the language model and also realizable according to grounded models of the environment. We frame this as a problem similar to probabilistic filtering: decode a sequence that both has high probability under the language model and high probability under a set of grounded model objectives. We demonstrate how such grounded models can be obtained across three simulation and real-world domains, and that the proposed decoding strategy is able to solve complex, long-horizon embodiment tasks in a robotic setting by leveraging the knowledge of both models. The project's website can be found at grounded-decoding.github.io.

翻译：近年来，大型语言模型（LLMs）的进展表明，通过自回归模型的预训练能够学习并利用互联网规模的知识。然而，将这些模型应用于具身智能体（如机器人）环境时面临挑战，原因在于这些模型缺乏对物理世界的经验、无法解析非语言观测数据，且忽视机器人可能需要的奖励或安全约束。另一方面，从交互数据中学习的语言条件化机器人策略虽能为智能体提供必要的现实世界基础，但由于训练数据的广度有限，此类策略缺少高层语义理解能力。因此，若要在保留语言模型语义知识的同时将其嵌入具身环境，必须构建既符合语言模型预测概率、又可通过环境基础模型实现的行动序列。我们将此问题类比为概率滤波：解码一个同时满足语言模型高概率分布与多组基础模型目标高概率分布的序列。通过三个模拟与真实世界领域的实验，我们展示了如何获取此类基础模型，并证明所提出的解码策略能够借助两种模型的知识，解决机器人环境中复杂的长期具身任务。项目网站见grounded-decoding.github.io。