While large language models (LLMs) have taken great strides towards helping humans with a plethora of tasks like search and summarization, hallucinations remain a major impediment towards gaining user trust. The fluency and coherence of model generations even when hallucinating makes it difficult to detect whether or not a model is hallucinating. In this work, we explore if the artifacts associated with the model generations can provide hints that the generation will contain hallucinations. Specifically, we probe LLMs at 1) the inputs via Integrated Gradients based token attribution, 2) the outputs via the Softmax probabilities, and 3) the internal state via self-attention and fully-connected layer activations for signs of hallucinations on open-ended question answering tasks. Our results show that the distributions of these artifacts differ between hallucinated and non-hallucinated generations. Building on this insight, we train binary classifiers that use these artifacts as input features to classify model generations into hallucinations and non-hallucinations. These hallucination classifiers achieve up to 0.80 AUROC. We further show that tokens preceding a hallucination can predict the subsequent hallucination before it occurs.

翻译：尽管大语言模型在搜索、摘要等诸多任务上取得了巨大进步，但幻觉现象仍是获取用户信任的主要障碍。即使模型在生成时出现幻觉，其输出的流畅性与连贯性也使检测是否产生幻觉变得困难。本研究探索模型生成过程中的关联特征能否为生成内容含幻觉提供提示信号。具体而言，我们通过：1）基于集成梯度的词元归因分析模型输入；2）基于Softmax概率分析模型输出；3）基于自注意力层和全连接层激活分析模型内部状态，在开放式问答任务中检测幻觉信号。实验结果表明，幻觉生成与非幻觉生成的这些特征分布存在差异。基于此发现，我们训练了以这些特征为输入的二分类器，将模型生成分类为幻觉与非幻觉。这些幻觉分类器的AUROC最高可达0.80。我们进一步证明，幻觉发生前的词元能够预测即将出现的后续幻觉。