Step-by-step decision planning with large language models (LLMs) is gaining attention in AI agent development. This paper focuses on decision planning with uncertainty estimation to address the hallucination problem in language models. Existing approaches are either white-box or computationally demanding, limiting use of black-box proprietary LLMs within budgets. The paper's first contribution is a non-parametric uncertainty quantification method for LLMs, efficiently estimating point-wise dependencies between input-decision on the fly with a single inference, without access to token logits. This estimator informs the statistical interpretation of decision trustworthiness. The second contribution outlines a systematic design for a decision-making agent, generating actions like ``turn on the bathroom light'' based on user prompts such as ``take a bath''. Users will be asked to provide preferences when more than one action has high estimated point-wise dependencies. In conclusion, our uncertainty estimation and decision-making agent design offer a cost-efficient approach for AI agent development.

翻译：基于大型语言模型（LLMs）的逐步决策规划在人工智能智能体开发中日益受到关注。本文聚焦于结合不确定性估计的决策规划方法，以应对语言模型中的幻觉问题。现有方法要么属于白盒模型，要么计算成本高昂，限制了在预算约束下使用黑盒专有LLMs。本文的第一个贡献是提出一种面向LLMs的非参数不确定性量化方法，无需访问令牌对数概率即可在单次推理中高效地在线估计输入-决策之间的逐点依赖关系。该估计量为决策可信度提供了统计解释依据。第二个贡献是提出了一种决策智能体的系统化设计框架，可基于用户提示（如“洗澡”）生成具体动作（如“打开浴室灯”）。当多个动作具有较高的估计逐点依赖值时，系统将提示用户提供偏好。综上，我们的不确定性估计与决策智能体设计方案为AI智能体开发提供了一种经济高效的途径。