Large language models are successful in answering factoid questions but are also prone to hallucination.We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics, an area not previously covered in studies on hallucinations.We are able to conduct this analysis via two key ideas.First, we identify the factual questions that query the same triplet knowledge but result in different answers. The difference between the model behaviors on the correct and incorrect outputs hence suggests the patterns when hallucinations happen. Second, to measure the pattern, we utilize mappings from the residual streams to vocabulary space. We reveal the different dynamics of the output token probabilities along the depths of layers between the correct and hallucinated cases. In hallucinated cases, the output token's information rarely demonstrates abrupt increases and consistent superiority in the later stages of the model. Leveraging the dynamic curve as a feature, we build a classifier capable of accurately detecting hallucinatory predictions with an 88\% success rate. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.

翻译：大语言模型在回答事实性问题方面表现出色，但也容易产生幻觉。我们从推理动力学角度研究了LLMs在拥有正确答案知识却仍产生幻觉的现象，这一视角在既往幻觉研究中尚未涉及。通过两个关键思想，我们得以进行此项分析：首先，我们识别出那些查询相同三元组知识却导致不同答案的事实性问题，模型在正确与错误输出上的行为差异因此揭示了幻觉发生的模式。其次，为量化该模式，我们利用残差流到词汇空间的映射，揭示了正确与幻觉情况下各层深度输出标记概率的不同动力学特征。在幻觉情况下，输出标记的信息在模型后期阶段鲜有突增且不具备持续优势。以动力学曲线为特征，我们构建的分类器能以88%的成功率准确检测幻觉性预测。本研究为理解LLMs在已知事实上产生幻觉的原因提供了新见解，更重要的是，为准确预测其是否正在产生幻觉提供了依据。