Retrieval-enhanced methods have become a primary approach in fact verification (FV); it requires reasoning over multiple retrieved pieces of evidence to verify the integrity of a claim. To retrieve evidence, existing work often employs off-the-shelf retrieval models whose design is based on the probability ranking principle. We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence. We introduce the feedback-based evidence retriever(FER) that optimizes the evidence retrieval process by incorporating feedback from the claim verifier. As a feedback signal we use the divergence in utility between how effectively the verifier utilizes the retrieved evidence and the ground-truth evidence to produce the final claim label. Empirical studies demonstrate the superiority of FER over prevailing baselines.
翻译:检索增强方法已成为事实核查的主要方法,它需要基于多条检索到的证据进行推理,以验证声明的真实性。为检索证据,现有工作通常采用基于概率排序原理设计的现成检索模型。我们认为,对于事实核查而言,需要关注的是声明验证者从检索证据中获得的效用,而非相关性。我们提出了基于反馈的证据检索器,该检索器通过引入声明验证者的反馈来优化证据检索过程。我们使用的反馈信号是:验证者利用检索证据与利用真实证据生成最终声明标签的效用差异。实证研究表明,我们的方法优于主流基线方法。