Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.

翻译：神经序列生成模型已知会"产生幻觉"，即生成与源文本无关的输出。这些幻觉可能具有危害性，但目前尚不清楚它们出现的条件以及如何减轻其影响。在本工作中，我们首先通过分析对比扰动源文本生成的幻觉与非幻觉输出中标记的相对贡献，识别出幻觉的内部模型症状。随后，我们证明这些症状是自然幻觉的可靠指标，并据此设计了一个轻量级幻觉检测器。该检测器在人工标注的英中和德英翻译测试集上，其表现优于无模型基线以及基于质量评估或大规模预训练模型的强分类器。