Automated fact-checking, using machine learning to verify claims, has grown vital as misinformation spreads beyond human fact-checking capacity. Large Language Models (LLMs) like GPT-4 are increasingly trusted to write academic papers, lawsuits, and news articles and to verify information, emphasizing their role in discerning truth from falsehood and the importance of being able to verify their outputs. Understanding the capacities and limitations of LLMs in fact-checking tasks is therefore essential for ensuring the health of our information ecosystem. Here, we evaluate the use of LLM agents in fact-checking by having them phrase queries, retrieve contextual data, and make decisions. Importantly, in our framework, agents explain their reasoning and cite the relevant sources from the retrieved context. Our results show the enhanced prowess of LLMs when equipped with contextual information. GPT-4 outperforms GPT-3, but accuracy varies based on query language and claim veracity. While LLMs show promise in fact-checking, caution is essential due to inconsistent accuracy. Our investigation calls for further research, fostering a deeper comprehension of when agents succeed and when they fail.

翻译：自动事实核查利用机器学习验证声明，在虚假信息传播超出人类核查能力时变得至关重要。GPT-4等大型语言模型（LLMs）日益被信赖用于撰写学术论文、法律文件、新闻文章以及验证信息，这凸显了其在辨别真伪中的作用，也强调了验证其输出内容的重要性。因此，理解LLMs在事实核查任务中的能力与局限对于确保信息生态系统的健康至关重要。本文通过让LLM代理自主构建查询、检索上下文数据并做出决策，评估其在事实核查中的应用。重要的是，在我们的框架中，代理会解释其推理过程并引用检索语境中的相关来源。结果表明，配备上下文信息的LLMs展现出更强的性能。GPT-4优于GPT-3，但准确性因查询语言和声明真实性而异。尽管LLMs在事实核查中展现出潜力，但由于准确性不稳定，仍需谨慎对待。我们的研究呼吁开展进一步探索，以深入理解代理成功与失败的情境。