While Language Agents have achieved promising success by placing Large Language Models at the core of a more versatile design that dynamically interacts with the external world, the existing approaches neglect the notion of uncertainty during these interactions. We present the Uncertainty-Aware Language Agent (UALA), a framework that orchestrates the interaction between the agent and the external world using uncertainty quantification. Compared with other well-known counterparts like ReAct, our extensive experiments across 3 representative tasks (HotpotQA, StrategyQA, MMLU) and various LLM sizes demonstrate that UALA brings a significant improvement of performance, while having a substantially lower reliance on the external world (i.e., reduced number of tool calls and tokens). Our analyses provide various insights including the great potential of UALA compared with agent fine-tuning, and underscore the unreliability of verbalised confidence of LLMs as a proxy for uncertainty.

翻译：尽管语言智能体通过将大型语言模型置于一个能够动态与外部世界交互的、设计更为通用的核心位置，已取得了令人瞩目的成功，但现有方法在这些交互过程中忽视了不确定性的概念。本文提出了不确定性感知语言智能体（UALA），这是一个利用不确定性量化来协调智能体与外部世界交互的框架。与ReAct等其他知名方法相比，我们在3个代表性任务（HotpotQA、StrategyQA、MMLU）上使用不同规模的LLM进行的广泛实验表明，UALA带来了显著的性能提升，同时对外部世界的依赖大幅降低（即减少了工具调用次数和令牌使用量）。我们的分析提供了多方面的见解，包括UALA相较于智能体微调的巨大潜力，并强调了LLM的口头化置信度作为不确定性代理的不可靠性。