Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hallucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

翻译：幻觉已被广泛认为是大型语言模型（LLMs）的重大缺陷。已有许多研究尝试减少幻觉程度，但这些工作至今大多基于经验方法，无法回答根本性问题——幻觉是否可以被完全消除。本文对该问题进行形式化，并证明LLMs中的幻觉无法被消除。具体而言，我们定义了一个形式化世界，其中幻觉被定义为可计算LLM与可计算真实函数之间的不一致性。通过运用学习理论的结果，我们证明LLMs无法学习所有可计算函数，因此将始终产生幻觉。由于形式化世界是更为复杂的真实世界的一部分，现实世界中的LLMs也同样无法避免幻觉。此外，针对受可证明时间复杂度约束的现实世界LLMs，我们描述了易产生幻觉的任务，并通过实验验证了我们的主张。最后，基于形式化世界框架，我们讨论了现有幻觉缓解方法的可能机制与效果，以及对LLMs安全部署的实践启示。