Retrieval-enhanced methods have become a primary approach in fact verification (FV); it requires reasoning over multiple retrieved pieces of evidence to verify the integrity of a claim. To retrieve evidence, existing work often employs off-the-shelf retrieval models whose design is based on the probability ranking principle. We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence. We introduce the feedback-based evidence retriever(FER) that optimizes the evidence retrieval process by incorporating feedback from the claim verifier. As a feedback signal we use the divergence in utility between how effectively the verifier utilizes the retrieved evidence and the ground-truth evidence to produce the final claim label. Empirical studies demonstrate the superiority of FER over prevailing baselines.
翻译:检索增强方法已成为事实核查领域的主要方法;它需要对多个检索到的证据进行推理以验证声明的完整性。为检索证据,现有工作通常采用基于概率排序原则设计的现成检索模型。我们认为,对于事实核查而言,我们应关注的并非相关性,而是声明验证器从检索证据中获得的实用性。我们引入了基于反馈的证据检索器,它通过整合来自声明验证器的反馈来优化证据检索过程。作为反馈信号,我们使用验证器利用检索证据与真实证据生成最终声明标签时所产生的效用差异。实证研究表明,该检索器优于当前主流基线模型。