Recent advances in large language models (LLMs) have empowered AI agents capable of performing various sequential decision-making tasks. However, effectively guiding LLMs to perform well in unfamiliar domains like web navigation, where they lack sufficient knowledge, has proven to be difficult with the demonstration-based in-context learning paradigm. In this paper, we introduce a novel framework, called AutoGuide, which addresses this limitation by automatically generating context-aware guidelines from offline experiences. Importantly, each context-aware guideline is expressed in concise natural language and follows a conditional structure, clearly describing the context where it is applicable. As a result, our guidelines facilitate the provision of relevant knowledge for the agent's current decision-making process, overcoming the limitations of the conventional demonstration-based learning paradigm. Our evaluation demonstrates that AutoGuide significantly outperforms competitive baselines in complex benchmark domains, including real-world web navigation.

翻译：近年来，大型语言模型（LLMs）的进展催生了能够执行各类序列决策任务的AI智能体。然而，在智能体缺乏足够知识的陌生领域（如网络导航）中，基于示例演示的上下文学习范式难以有效引导LLMs取得良好表现。本文提出名为AutoGuide的新型框架，通过从离线经验中自动生成上下文感知指导原则来解决这一局限。值得注意的是，每条上下文感知指导原则均以简洁的自然语言表达，并遵循条件式结构，清晰描述其适用情境。因此，我们的指导原则能够为智能体当前决策过程提供相关知识，克服了传统基于演示的学习范式的局限性。评估结果表明，在包括真实网络导航在内的复杂基准领域中，AutoGuide显著优于现有竞争基线方法。