Autonomous systems are soon to be ubiquitous, from manufacturing autonomy to agricultural field robots, and from health care assistants to the entertainment industry. The majority of these systems are developed with modular sub-components for decision-making, planning, and control that may be hand-engineered or learning-based. While these existing approaches have been shown to perform well under the situations they were specifically designed for, they can perform especially poorly in rare, out-of-distribution scenarios that will undoubtedly arise at test-time. The rise of foundation models trained on multiple tasks with impressively large datasets from a variety of fields has led researchers to believe that these models may provide common sense reasoning that existing planners are missing. Researchers posit that this common sense reasoning will bridge the gap between algorithm development and deployment to out-of-distribution tasks, like how humans adapt to unexpected scenarios. Large language models have already penetrated the robotics and autonomous systems domains as researchers are scrambling to showcase their potential use cases in deployment. While this application direction is very promising empirically, foundation models are known to hallucinate and generate decisions that may sound reasonable, but are in fact poor. We argue there is a need to step back and simultaneously design systems that can quantify the certainty of a model's decision, and detect when it may be hallucinating. In this work, we discuss the current use cases of foundation models for decision-making tasks, provide a general definition for hallucinations with examples, discuss existing approaches to hallucination detection and mitigation with a focus on decision problems, and explore areas for further research in this exciting field.

翻译：自主系统即将无处不在，从制造自动化到农业田间机器人，从医疗辅助到娱乐产业。这些系统大多由模块化子组件构成，用于决策、规划与控制，可能为手工设计或基于学习方法。尽管现有方法在其专门设计的场景下表现良好，但在测试时必然出现的罕见分布外场景中可能表现极差。基于多任务训练、使用来自不同领域的大规模数据集的基础模型兴起，使研究者相信这些模型能提供现有规划器所缺失的常识推理能力。研究者认为，这种常识推理将弥合算法开发与分布外任务部署之间的差距，正如人类适应意外场景的方式。随着研究者争相展示大型语言模型在部署中的潜在用例，该模型已渗透到机器人和自主系统领域。尽管这一应用方向在经验上极具前景，但基础模型已知会产生幻觉，生成看似合理实则低劣的决策。我们认为有必要退一步，同时设计能量化模型决策确定性并检测其可能产生幻觉的系统。本文探讨了当前基础模型在决策任务中的使用案例，为幻觉提供了包含示例的通用定义，讨论了现有幻觉检测与缓解方法（聚焦于决策问题），并探索了这一前沿领域的进一步研究方向。