Verifying fact-checking claims poses a significant challenge, even for humans. Recent approaches have demonstrated that decomposing claims into relevant questions to gather evidence enhances the efficiency of the fact-checking process. In this paper, we provide empirical evidence showing that this question decomposition can be effectively automated. We demonstrate that smaller generative models, fine-tuned for the question generation task using data augmentation from various datasets, outperform large language models by up to 8%. Surprisingly, in some cases, the evidence retrieved using machine-generated questions proves to be significantly more effective for fact-checking than that obtained from human-written questions. We also perform manual evaluation of the decomposed questions to assess the quality of the questions generated.
翻译:验证事实核查声明是一项重大挑战,即使对人类而言也是如此。近期研究表明,将声明分解为相关问题以收集证据,能够提高事实核查过程的效率。本文通过实证证据表明,这种问题分解过程可以有效实现自动化。我们证明,针对问题生成任务进行微调的小型生成模型,通过利用来自不同数据集的数据增强,其性能优于大型语言模型高达8%。令人惊讶的是,在某些情况下,使用机器生成问题检索到的证据对于事实核查的效果,显著优于基于人工撰写问题所获得的证据。我们还对分解后的问题进行了人工评估,以评判所生成问题的质量。