Large language models (LLMs) have emerged as pivotal contributors in contemporary natural language processing and are increasingly being applied across a diverse range of industries. However, these large-scale probabilistic statistical models cannot currently ensure the requisite quality in professional content generation. These models often produce hallucinated text, compromising their practical utility in professional contexts. To assess the authentic reliability of LLMs in text generation, numerous initiatives have developed benchmark evaluations for hallucination phenomena. Nevertheless, these benchmarks frequently utilize constrained generation techniques due to cost and temporal constraints. These techniques encompass the use of directed hallucination induction and strategies that deliberately alter authentic text to produce hallucinations. These approaches are not congruent with the unrestricted text generation demanded by real-world applications. Furthermore, a well-established Chinese-language dataset dedicated to the evaluation of hallucinations in text generation is presently lacking. Consequently, we have developed an Unconstrained Hallucination Generation Evaluation (UHGEval) benchmark, designed to compile outputs produced with minimal restrictions by LLMs. Concurrently, we have established a comprehensive benchmark evaluation framework to aid subsequent researchers in undertaking scalable and reproducible experiments. We have also executed extensive experiments, evaluating prominent Chinese language models and the GPT series models to derive professional performance insights regarding hallucination challenges.

翻译：大语言模型已成为当代自然语言处理领域的关键贡献者，并正日益广泛应用于各行各业。然而，这些大规模概率统计模型目前尚无法确保在专业内容生成中达到所需的质量标准。这些模型常常产生幻觉文本，从而损害了其在专业场景中的实际效用。为评估大语言模型在文本生成中的真实可靠性，众多研究已针对幻觉现象开发了基准评测。然而，由于成本与时间限制，这些基准评测常采用约束生成技术。这些技术包括使用定向幻觉诱导策略，以及刻意篡改真实文本来产生幻觉的方法。这些方式与实际应用所要求的无约束文本生成模式并不相符。此外，目前尚缺乏专门用于评估文本生成幻觉的成熟中文数据集。为此，我们开发了无约束幻觉生成评估基准，旨在收集大语言模型在最小限制条件下产生的输出。同时，我们建立了一个全面的基准评测框架，以帮助后续研究者进行可扩展、可复现的实验。我们还进行了广泛的实验，评估了主流的中文语言模型以及GPT系列模型，以获取关于幻觉问题的专业性能洞察。