Large language models (LLMs), such as ChatGPT, are prone to generate hallucinations, \ie content that conflicts with the source or cannot be verified by the factual knowledge. To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation for Large Language Models (HELMA) benchmark, a large collection of generated and human-annotated hallucinated samples for evaluating the performance of LLMs in recognizing and alleviating hallucination. To generate these samples, we propose a ChatGPT-based two-step framework, \ie sampling-then-filtering. Specifically, we first adopt two different sampling methods to generate hallucinated samples based on instructions, and then use an example-enhanced filtering method to select the best one. Furthermore, we also hire some human labelers to annotate the hallucinations in ChatGPT responses. The empirical results suggest that ChatGPT has some probabilities to generate hallucinations and existing LLMs face great challenges in recognizing the hallucinations in text. In addition, the performance can be improved by providing external knowledge or adding reasoning steps. Our benchmark can be accessed at https://github.com/RUCAIBox/HELMA.

翻译：大型语言模型（如ChatGPT）易产生幻觉现象，即生成与事实知识相矛盾或无法通过事实知识验证的内容。为理解语言模型在何种内容类型及程度上易产生幻觉，我们提出大型语言模型幻觉评估基准（HELMA）。该基准包含大量通过生成与人工标注形成的幻觉样本，用于评估语言模型识别与缓解幻觉的能力。为生成这些样本，我们提出基于ChatGPT的两阶段框架（采样-过滤法）：首先采用两种不同采样方法基于指令生成幻觉样本，随后通过示例增强过滤法筛选最优样本。此外，我们雇佣人工标注员对ChatGPT回复中的幻觉进行标注。实验结果表明：ChatGPT存在一定幻觉生成概率，现有语言模型在识别文本幻觉方面面临重大挑战，而引入外部知识或增加推理步骤可提升性能。本基准访问地址为https://github.com/RUCAIBox/HELMA。