Claim verification is a core component of automated fact-checking systems, aimed at determining the truthfulness of a statement by assessing it against reliable evidence sources such as documents or knowledge bases. This work presents KG-CRAFT, a method that improves automatic claim verification by leveraging large language models (LLMs) augmented with contrastive questions grounded in a knowledge graph. KG-CRAFT first constructs a knowledge graph from claims and associated reports, then formulates contextually relevant contrastive questions based on the knowledge graph structure. These questions guide the distillation of evidence-based reports, which are synthesised into a concise summary that is used for veracity assessment by LLMs. Extensive evaluations on two real-world datasets (LIAR-RAW and RAWFC) demonstrate that our method achieves a new state-of-the-art in predictive performance. Comprehensive analyses validate in detail the effectiveness of our knowledge graph-based contrastive reasoning approach in improving LLMs' fact-checking capabilities.
翻译:声明验证是自动事实核查系统的核心组成部分,旨在通过将陈述与文档或知识库等可靠证据源进行比对,以判定其真实性。本研究提出KG-CRAFT方法,该方法通过利用大语言模型(LLMs)并结合基于知识图谱构建的对比性问题,以改进自动声明验证。KG-CRAFT首先从声明及相关报告中构建知识图谱,随后基于知识图谱结构生成上下文相关的对比性问题。这些问题引导基于证据的报告提炼,进而综合成简洁摘要,供LLMs用于真实性评估。在两个真实世界数据集(LIAR-RAW和RAWFC)上的广泛评估表明,我们的方法在预测性能上达到了新的最先进水平。综合分析详细验证了我们基于知识图谱的对比推理方法在提升LLMs事实核查能力方面的有效性。