Large language models (LLMs) have emerged as pivotal contributors in contemporary natural language processing and are increasingly being applied across a diverse range of industries. However, these large-scale probabilistic statistical models cannot currently ensure the requisite quality in professional content generation. These models often produce hallucinated text, compromising their practical utility in professional contexts. To assess the authentic reliability of LLMs in text generation, numerous initiatives have developed benchmark evaluations for hallucination phenomena. Nevertheless, these benchmarks frequently utilize constrained generation techniques due to cost and temporal constraints. These techniques encompass the use of directed hallucination induction and strategies that deliberately alter authentic text to produce hallucinations. These approaches are not congruent with the unrestricted text generation demanded by real-world applications. Furthermore, a well-established Chinese-language dataset dedicated to the evaluation of hallucinations in text generation is presently lacking. Consequently, we have developed an Unconstrained Hallucination Generation Evaluation (UHGEval) benchmark, designed to compile outputs produced with minimal restrictions by LLMs. Concurrently, we have established a comprehensive benchmark evaluation framework to aid subsequent researchers in undertaking scalable and reproducible experiments. We have also executed extensive experiments, evaluating prominent Chinese language models and the GPT series models to derive professional performance insights regarding hallucination challenges.

翻译：大语言模型已成为当代自然语言处理的关键贡献者，并正被广泛应用于各个行业。然而，这些大规模概率统计模型目前无法确保专业内容生成中所需的质量。这些模型常产生幻觉文本，削弱了其在专业场景中的实用性。为评估大语言模型在文本生成中的真实可靠性，众多研究项目已开发针对幻觉现象的基准评估。然而，这些基准因成本和时间限制，常采用受约束生成技术，包括定向诱导幻觉以及故意篡改真实文本以产生幻觉的策略。这些方法无法与现实应用所需的无约束文本生成相契合。此外，当前尚缺乏成熟的中文语言数据集用于专门评估文本生成中的幻觉问题。为此，我们构建了无约束幻觉生成评估基准，旨在收集大语言模型在最小限制条件下生成的输出。同时，我们建立了全面的基准评估框架，以支持后续研究者开展可扩展、可重复的实验。我们还执行了广泛的实验，评估了主流中文语言模型及GPT系列模型，以获取关于幻觉挑战的专业性能见解。