Despite their impressive capabilities, large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information, a phenomenon commonly known as ``hallucination''. In this work, we propose a simple \textit{Induce-then-Contrast} Decoding (ICD) strategy to alleviate hallucinations. We first construct a factually weak LLM by inducing hallucinations from the original LLMs. Then, we penalize these induced hallucinations during decoding to enhance the factuality of the generated content. Concretely, we determine the final next-token predictions by amplifying the predictions from the original model and downplaying the induced untruthful predictions via contrastive decoding. Experimental results on both discrimination-based and generation-based hallucination evaluation benchmarks, such as TruthfulQA and \textsc{FActScore}, demonstrate that our proposed ICD methods can effectively enhance the factuality of LLMs across various model sizes and families. For example, when equipped with ICD, Llama2-7B-Chat and Mistral-7B-Instruct achieve performance comparable to ChatGPT and GPT4 on TruthfulQA, respectively.

翻译：尽管大型语言模型（LLMs）展现出令人印象深刻的性能，但观察发现它们会生成包含不准确或虚构信息的响应，这种现象通常被称为"幻觉"。在本工作中，我们提出一种简单的“诱导-对比”解码（ICD）策略来缓解幻觉。首先，我们通过从原始LLMs中诱导幻觉构建一个事实性薄弱的LLM。然后，在解码过程中对这些诱导产生的幻觉进行惩罚，以增强生成内容的事实准确性。具体而言，我们通过放大原始模型的预测并削弱诱导出的不真实预测（通过对比解码）来确定最终的下一个词预测结果。在基于判别和基于生成的幻觉评估基准（如TruthfulQA和FActScore）上的实验结果表明，我们提出的ICD方法能够有效增强不同模型规模和系列的事实准确性。例如，当配备ICD时，Llama2-7B-Chat和Mistral-7B-Instruct在TruthfulQA上分别达到了与ChatGPT和GPT4相当的性能。