The rapid advancement of large language model(LLM) technology has facilitated its integration into various domains of professional and daily life. However, the persistent challenge of LLM hallucination has emerged as a critical limitation, significantly compromising the reliability and trustworthiness of AI-generated content. This challenge has garnered significant attention within the scientific community, prompting extensive research efforts in hallucination detection and mitigation strategies. Current methodological frameworks reveal a critical limitation: traditional uncertainty quantification approaches demonstrate effectiveness primarily within conventional question-answering paradigms, yet exhibit notable deficiencies when confronted with non-canonical or adversarial questioning strategies. This performance gap raises substantial concerns regarding the dependability of LLM responses in real-world applications requiring robust critical thinking capabilities. This study aims to fill this gap by proposing an uncertainty quantification scenario in the task of generating with multiple facts. We have meticulously constructed a set of trap questions contained with fake names. Based on this scenario, we innovatively propose a novel and robust uncertainty quantification method(RU). A series of experiments have been conducted to verify its effectiveness. The results show that the constructed set of trap questions performs excellently. Moreover, when compared with the baseline methods on four different models, our proposed method has demonstrated great performance, with an average increase of 0.1-0.2 in ROCAUC values compared to the best performing baseline method, providing new sights and methods for addressing the hallucination issue of LLMs.

翻译：大型语言模型（LLM）技术的快速发展促进了其在专业和日常生活各个领域的融合。然而，LLM幻觉的持续存在已成为一个关键限制，显著损害了AI生成内容的可靠性和可信度。这一挑战已引起科学界的广泛关注，推动了在幻觉检测和缓解策略方面的大量研究工作。当前的方法论框架揭示了一个关键局限：传统的不确定性量化方法主要在常规问答范式中表现出有效性，但在面对非规范或对抗性提问策略时则表现出明显不足。这一性能差距引发了人们对LLM在需要强大批判性思维能力的实际应用中的响应可靠性的重大担忧。本研究旨在通过提出一种在生成包含多重事实的任务中的不确定性量化场景来填补这一空白。我们精心构建了一组包含虚假名称的陷阱问题。基于此场景，我们创新性地提出了一种新颖且鲁棒的不确定性量化方法（RU）。我们进行了一系列实验以验证其有效性。结果表明，所构建的陷阱问题集表现优异。此外，在四个不同模型上与基线方法进行比较时，我们提出的方法展现了卓越的性能，其ROCAUC值相较于表现最佳的基线方法平均提高了0.1-0.2，为解决LLM的幻觉问题提供了新的视角和方法。