Numerous large language model (LLM) agents have been built for different tasks like web navigation and online shopping due to LLM's wide knowledge and text-understanding ability. Among these works, many of them utilize in-context examples to achieve generalization without the need for fine-tuning, while few of them have considered the problem of how to select and effectively utilize these examples. Recently, methods based on trajectory-level retrieval with task meta-data and using trajectories as in-context examples have been proposed to improve the agent's overall performance in some sequential decision making tasks. However, these methods can be problematic due to plausible examples retrieved without task-specific state transition dynamics and long input with plenty of irrelevant context. In this paper, we propose a novel framework (TRAD) to address these issues. TRAD first conducts Thought Retrieval, achieving step-level demonstration selection via thought matching, leading to more helpful demonstrations and less irrelevant input noise. Then, TRAD introduces Aligned Decision, complementing retrieved demonstration steps with their previous or subsequent steps, which enables tolerance for imperfect thought and provides a choice for balance between more context and less noise. Extensive experiments on ALFWorld and Mind2Web benchmarks show that TRAD not only outperforms state-of-the-art models but also effectively helps in reducing noise and promoting generalization. Furthermore, TRAD has been deployed in real-world scenarios of a global business insurance company and improves the success rate of robotic process automation.

翻译：由于大语言模型（LLM）具备广泛的知识和文本理解能力，研究者已构建了众多LLM智能体以应对网页导航、在线购物等不同任务。这些工作中，许多通过利用上下文示例实现泛化而无需微调，但鲜有工作关注如何选择并有效利用这些示例。近年来，基于任务元数据进行轨迹级检索并将轨迹作为上下文示例的方法被提出，以提升智能体在部分序列决策任务中的整体性能。然而，这些方法可能因检索到的示例缺乏任务特定状态转移动力学且输入过长包含大量无关上下文而存在问题。本文提出一个新框架（TRAD）以解决这些问题。TRAD首先进行思维检索（Thought Retrieval），通过思维匹配实现步骤级示范选择，从而获得更有效的示范并减少无关输入噪声。随后，TRAD引入对齐决策（Aligned Decision），将检索到的示范步骤与其前后步骤进行互补，这既能容忍不完美的思维，又能为平衡更多上下文与更少噪声提供选择。在ALFWorld和Mind2Web基准上的大量实验表明，TRAD不仅优于现有最优模型，还能有效减少噪声并促进泛化。此外，TRAD已部署于一家全球商业保险公司的真实场景中，提升了机器人流程自动化的成功率。