The escalating challenge of misinformation, particularly in the context of political discourse, necessitates advanced solutions for fact-checking. We introduce innovative approaches to enhance the reliability and efficiency of multimodal fact-checking through the integration of Large Language Models (LLMs) with Retrieval-augmented Generation (RAG)- based advanced reasoning techniques. This work proposes two novel methodologies, Chain of RAG (CoRAG) and Tree of RAG (ToRAG). The approaches are designed to handle multimodal claims by reasoning the next questions that need to be answered based on previous evidence. Our approaches improve the accuracy of veracity predictions and the generation of explanations over the traditional fact-checking approach of sub-question generation with chain of thought veracity prediction. By employing multimodal LLMs adept at analyzing both text and images, this research advances the capability of automated systems in identifying and countering misinformation.

翻译：虚假信息挑战日益严峻，尤其在政治话语语境中，亟需先进的事实核查解决方案。我们通过将大语言模型与基于检索增强生成的高级推理技术相结合，提出了创新方法以提升多模态事实核查的可靠性与效率。本文提出两种新方法：链式RAG与树状RAG。这些方法通过根据已有证据推理需回答的下一个问题，专门处理多模态声明。相较于传统子问题生成结合思维链真实性预测的事实核查方法，我们的方法显著提升了真实性预测的准确性与解释生成质量。通过采用能够同时分析文本与图像的多模态大语言模型，本研究推动了自动化系统在识别与对抗虚假信息领域的能力发展。