Fact-checking aims to verify the truthfulness of a claim based on the retrieved evidence. Existing methods typically follow a decomposition paradigm, in which a claim is broken down into sub-claims that are individually verified. However, the decomposition paradigm may introduce noise to the verification process due to irrelevant entities or evidence, ultimately degrading verification accuracy. To address this problem, we propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs). Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them. Then, it refines the retrieved evidence based on the claim to reduce irrelevant information. Finally, it calibrates the verification process by re-evaluating low-confidence predictions. Experiments on two popular fact-checking datasets (HOVER and FEVEROUS-S) demonstrate that our framework achieves superior performance compared with competitive baselines.
翻译:事实核查旨在基于检索到的证据验证声明的真实性。现有方法通常遵循分解范式,即将声明分解为多个子声明分别进行验证。然而,分解范式可能因引入不相关实体或证据而为验证过程带来噪声,最终降低验证准确性。为解决该问题,我们提出一种基于大语言模型(LLMs)的检索-精炼-校准(RRC)框架。具体而言,该框架首先识别声明中提及的实体并检索相关证据;随后基于声明对检索证据进行精炼以减少无关信息;最后通过重新评估低置信度预测来校准验证过程。在两个主流事实核查数据集(HOVER与FEVEROUS-S)上的实验表明,相较于现有竞争性基线方法,本框架实现了更优的性能。