Retrieval-augmented language models have exhibited promising performance across various areas of natural language processing (NLP), including fact-critical tasks. However, due to the black-box nature of advanced large language models (LLMs) and the non-retrieval-oriented supervision signal of specific tasks, the training of retrieval model faces significant challenges under the setting of black-box LLM. We propose an approach leveraging Fine-grained Feedback with Reinforcement Retrieval (FFRR) to enhance fact-checking on news claims by using black-box LLM. FFRR adopts a two-level strategy to gather fine-grained feedback from the LLM, which serves as a reward for optimizing the retrieval policy, by rating the retrieved documents based on the non-retrieval ground truth of the task. We evaluate our model on two public datasets for real-world news claim verification, and the results demonstrate that FFRR achieves significant improvements over strong LLM-enabled and non-LLM baselines.

翻译：摘要：检索增强型语言模型在自然语言处理（NLP）的多个领域（包括事实关键型任务）展现出显著性能。然而，由于先进大型语言模型（LLM）的黑箱特性及特定任务缺乏面向检索优化的监督信号，在黑盒LLM场景下训练检索模型面临重大挑战。我们提出一种基于细粒度反馈的强化检索方法（FFRR），通过利用黑盒大语言模型增强新闻声明的事实核查能力。FFRR采用双层策略从LLM中收集细粒度反馈，该反馈基于任务的非检索式真实标注对检索文档进行评级，从而作为优化检索策略的奖励信号。我们在两个公开真实新闻声明验证数据集上评估模型，结果表明FFRR相较于强LLM基线及非LLM基线方法均取得了显著性能提升。