Despite their impressive capabilities, large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information, a phenomenon commonly known as ``hallucination''. In this work, we propose a simple \textit{Induce-then-Contrast} Decoding (ICD) strategy to alleviate hallucinations. We first construct a factually weak LLM by inducing hallucinations from the original LLMs. Then, we penalize these induced hallucinations during decoding to enhance the factuality of the generated content. Concretely, we determine the final next-token predictions by amplifying the predictions from the original model and downplaying the induced untruthful predictions via contrastive decoding. Experimental results on both discrimination-based and generation-based hallucination evaluation benchmarks, such as TruthfulQA and \textsc{FActScore}, demonstrate that our proposed ICD methods can effectively enhance the factuality of LLMs across various model sizes and families. For example, when equipped with ICD, Llama2-7B-Chat and Mistral-7B-Instruct achieve performance comparable to ChatGPT and GPT4 on TruthfulQA, respectively.

翻译：尽管大语言模型（LLMs）展现出令人印象深刻的能力，但人们观察到它们会生成包含不准确或虚构信息的回答——这种现象通常被称为“幻觉”。在本研究中，我们提出一种简单的“诱导-对比”解码（ICD）策略来缓解幻觉问题。首先，我们通过从原始LLM中诱导出幻觉来构建一个事实性较弱的LLM；然后，在解码过程中对这些诱导出的幻觉进行惩罚，以增强生成内容的事实性。具体地，我们通过放大原始模型的预测结果并借助对比解码弱化诱导出的不真实预测，来确定最终的下一词预测。在基于判别和基于生成的幻觉评估基准（如TruthfulQA和FActScore）上的实验结果表明，我们提出的ICD方法能够有效提升不同规模与系列的大语言模型的事实性。例如，在使用ICD方法时，Llama2-7B-Chat和Mistral-7B-Instruct在TruthfulQA上分别达到了与ChatGPT和GPT4相当的性能。