Large Language Models (LLMs) demonstrate remarkable capabilities in text understanding and generation. However, their tendency to produce factually inconsistent outputs, commonly referred to as ''hallucinations'', remains a critical challenge. Existing approaches, such as retrieval-based and inference-time correction methods, primarily address this issue at the input or output level, often overlooking the intrinsic information refinement process and the role of premature layers. Meanwhile, alignment- and fine-tuning-based methods are resource-intensive. In this paper, we propose PLI (Premature Layers Interpolation), a novel, training-free, and plug-and-play intervention designed to enhance factuality. PLI mitigates hallucinations by inserting premature layers formed through mathematical interpolation with adjacent layers. Inspired by stable diffusion and sampling steps, PLI extends the depth of information processing and transmission in LLMs, improving factual coherence. Experiments on four publicly available datasets demonstrate that PLI effectively reduces hallucinations while outperforming existing baselines in most cases. Further analysis suggests that the success of layer interpolation is closely linked to LLMs' internal mechanisms. Our dataset and code are available at https://github.com/CuSO4-Chen/PLI.
翻译:大语言模型(LLM)在文本理解与生成方面展现出卓越能力。然而,其产生事实不一致输出(通常称为"幻觉")的倾向仍是关键挑战。现有方法(如基于检索和推理时校正的方法)主要在输入或输出层面解决此问题,往往忽略了内在信息精化过程及早期层的作用。同时,基于对齐和微调的方法资源消耗较大。本文提出PLI(前置层插值),这是一种新颖、免训练、即插即用的干预机制,旨在增强事实性。PLI通过插入由相邻层数学插值形成的早期层来缓解幻觉现象。受稳定扩散和采样步骤启发,PLI扩展了LLM中信息处理与传递的深度,从而提升事实连贯性。在四个公开数据集上的实验表明,PLI能有效减少幻觉,且在多数情况下优于现有基线方法。进一步分析表明,层插值的成功与LLM的内部机制密切相关。我们的数据集与代码公开于https://github.com/CuSO4-Chen/PLI。