The primary limitation of large language models (LLMs) is their restricted understanding of the world. This poses significant difficulties for LLM-based agents, particularly in domains where pre-trained LLMs lack sufficient knowledge. In this paper, we introduce a novel framework, called AutoGuide, that bridges the knowledge gap in pre-trained LLMs by leveraging implicit knowledge in offline experiences. Specifically, AutoGuide effectively extracts knowledge embedded in offline data by extracting a set of state-aware guidelines. Importantly, each state-aware guideline is expressed in concise natural language and follows a conditional structure, clearly describing the state where it is applicable. As such, the resulting guidelines enable a principled way to provide helpful knowledge pertinent to an agent's current decision-making process. We show that our approach outperforms competitive LLM-based baselines by a large margin in sequential decision-making benchmarks.

翻译：摘要：大语言模型（LLM）的主要局限性在于其对世界的理解能力受限。这给基于LLM的智能体带来了显著困难，尤其是在预训练LLM缺乏足够知识的领域中。本文提出了一种名为AutoGuide的新型框架，通过利用离线经验中的隐式知识来弥合预训练LLM的知识鸿沟。具体而言，AutoGuide通过提取一组状态感知指南，有效抽取离线数据中蕴含的知识。重要的是，每条状态感知指南均以简洁的自然语言表述并遵循条件式结构，清晰描述了其适用的状态。因此，生成的指南能够以原则性的方式提供与智能体当前决策过程相关的有用知识。我们证明，在序列决策基准测试中，本方法以显著优势超越了具有竞争力的基于LLM的基线方法。