Automatic fact-checking aims to support professional fact-checkers by offering tools that can help speed up manual fact-checking. Yet, existing frameworks fail to address the key step of producing output suitable for broader dissemination to the general public: while human fact-checkers communicate their findings through fact-checking articles, automated systems typically produce little or no justification for their assessments. Here, we aim to bridge this gap. In particular, we argue for the need to extend the typical automatic fact-checking pipeline with automatic generation of full fact-checking articles. We first identify key desiderata for such articles through a series of interviews with experts from leading fact-checking organizations. We then develop QRAFT, an LLM-based agentic framework that mimics the writing workflow of human fact-checkers. Finally, we assess the practical usefulness of QRAFT through human evaluations with professional fact-checkers. Our evaluation shows that while QRAFT outperforms several previously proposed text-generation approaches, it lags considerably behind expert-written articles. We hope that our work will enable further research in this new and important direction. The code for our implementation is available at https://github.com/mbzuai-nlp/qraft.git.

翻译：自动事实核查旨在通过提供能够加速人工核查流程的工具来支持专业事实核查人员。然而，现有框架未能解决生成适合向公众广泛传播的输出这一关键步骤：人类核查员通过事实核查文章传达其发现，而自动化系统通常对其评估结果提供极少甚至不提供解释。本研究致力于弥合这一差距。我们特别主张需要扩展典型自动事实核查流程，实现完整事实核查文章的自动生成。我们首先通过对多家领先事实核查机构专家的系列访谈，确定了此类文章的核心要求。随后开发了QRAFT——一个基于大型语言模型的智能体框架，该框架模拟人类事实核查员的写作工作流。最后，我们通过专业事实核查人员的人工评估来检验QRAFT的实际效用。评估结果表明，虽然QRAFT优于先前提出的多种文本生成方法，但与专家撰写的文章相比仍存在显著差距。我们希望这项工作能推动这一新兴重要方向的后续研究。实现代码已发布于https://github.com/mbzuai-nlp/qraft.git。