Despite tremendous advancements in large language models (LLMs) over recent years, a notably urgent challenge for their practical deployment is the phenomenon of hallucination, where the model fabricates facts and produces non-factual statements. In response, we propose PoLLMgraph, a Polygraph for LLMs, as an effective model-based white-box detection and forecasting approach. PoLLMgraph distinctly differs from the large body of existing research that concentrates on addressing such challenges through black-box evaluations. In particular, we demonstrate that hallucination can be effectively detected by analyzing the LLM's internal state transition dynamics during generation via tractable probabilistic models. Experimental results on various open-source LLMs confirm the efficacy of PoLLMgraph, outperforming state-of-the-art methods by a considerable margin, evidenced by over 20% improvement in AUC-ROC on common benchmarking datasets like TruthfulQA. Our work paves a new way for model-based white-box analysis of LLMs, motivating the research community to further explore, understand, and refine the intricate dynamics of LLM behaviors.

翻译：尽管近年来大语言模型（LLMs）取得了巨大进步，但其实际部署面临一个尤为紧迫的挑战——幻觉现象，即模型虚构事实并产生不实陈述。为此，我们提出PoLLMgraph（大语言模型测谎仪），这是一种基于模型的白盒检测与预测方法。与现有大量专注于通过黑盒评估解决该挑战的研究截然不同，我们证明：通过可处理的概率模型分析LLM在生成过程中的内部状态转换动力学，即可有效检测幻觉。针对多个开源LLM的实验结果证实了PoLLMgraph的效力——在TruthfulQA等通用基准数据集上，其AUC-ROC指标较当前最优方法实现了超过20%的提升。本研究为基于模型的大语言模型白盒分析开辟了新路径，有望激励学界进一步探索、理解并优化LLM行为的复杂动力学机制。