Despite their impressive generative capabilities, LLMs are hindered by fact-conflicting hallucinations in real-world applications. The accurate identification of hallucinations in texts generated by LLMs, especially in complex inferential scenarios, is a relatively unexplored area. To address this gap, we present FactCHD, a dedicated benchmark designed for the detection of fact-conflicting hallucinations from LLMs. FactCHD features a diverse dataset that spans various factuality patterns, including vanilla, multi-hop, comparison, and set operation. A distinctive element of FactCHD is its integration of fact-based evidence chains, significantly enhancing the depth of evaluating the detectors' explanations. Experiments on different LLMs expose the shortcomings of current approaches in detecting factual errors accurately. Furthermore, we introduce Truth-Triangulator that synthesizes reflective considerations by tool-enhanced ChatGPT and LoRA-tuning based on Llama2, aiming to yield more credible detection through the amalgamation of predictive results and evidence. The benchmark dataset is available at https://github.com/zjunlp/FactCHD.

翻译：尽管大语言模型具备令人印象深刻的生成能力，但在实际应用中仍受事实冲突性幻觉的阻碍。准确识别大语言模型生成文本中的幻觉，尤其是在复杂推理场景下，仍是一个相对未被充分探索的领域。为弥补这一空白，我们提出了FactCHD，一个专为大语言模型事实冲突性幻觉检测设计的基准测试。FactCHD包含一个多样化的数据集，涵盖多种事实性模式，包括基础型、多跳型、比较型和集合操作型。FactCHD的一个显著特点是其集成了基于事实的证据链，这显著增强了评估检测器解释深度的能力。在不同大语言模型上的实验揭示了当前方法在准确检测事实错误方面的不足。此外，我们提出了Truth-Triangulator方法，该方法综合了工具增强型ChatGPT的反思性考量与基于Llama2的LoRA微调，旨在通过融合预测结果与证据来产生更可信的检测。基准测试数据集可在 https://github.com/zjunlp/FactCHD 获取。