Real-world misinformation can be partially correct and even factual but misleading. It undermines public trust in science and democracy, particularly on social media, where it can spread rapidly. High-quality and timely correction of misinformation that identifies and explains its (in)accuracies has been shown to effectively reduce false beliefs. Despite the wide acceptance of manual correction, it is difficult to be timely and scalable, a concern as technologies like large language models (LLMs) make misinformation easier to produce. LLMs also have versatile capabilities that could accelerate misinformation correction-however, they struggle due to a lack of recent information, a tendency to produce false content, and limitations in addressing multimodal information. We propose MUSE, an LLM augmented with access to and credibility evaluation of up-to-date information. By retrieving evidence as refutations or contexts, MUSE identifies and explains (in)accuracies in a piece of content-not presupposed to be misinformation-with references. It also describes images and conducts multimodal searches to verify and correct multimodal content. Fact-checking experts evaluate responses to social media content that are not presupposed to be (non-)misinformation but broadly include incorrect, partially correct, and correct posts, that may or may not be misleading. We propose and evaluate 13 dimensions of misinformation correction quality, ranging from the accuracy of identifications and factuality of explanations to the relevance and credibility of references. The results demonstrate MUSE's ability to promptly write high-quality responses to potential misinformation on social media-overall, MUSE outperforms GPT-4 by 37% and even high-quality responses from laypeople by 29%. This work reveals LLMs' potential to help combat real-world misinformation effectively and efficiently.
翻译:现实世界中的错误信息可能部分正确甚至符合事实但具有误导性。这会削弱公众对科学和民主的信任,尤其是在社交媒体上,错误信息可能快速传播。高质量且及时的纠错——识别并解释其中的(不)准确性——已被证明能有效减少错误信念。尽管人工纠正被广泛接受,但其难以做到及时和可扩展,而大型语言模型(LLM)等技术更使得错误信息易于生成,令人担忧。LLM也具备加速错误信息纠正的多种能力——然而,它们因缺乏最新信息、易产生虚假内容以及处理多模态信息的局限性而面临困难。我们提出MUSE,这是一种增强了对最新信息访问和可信度评估的LLM。通过检索证据作为反驳或上下文,MUSE能识别并解释内容中的(不)准确性——且不预设其为错误信息——并附上参考文献。它还能描述图像并进行多模态搜索,以验证和纠正多模态内容。事实核查专家评估对社交媒体内容的回应,这些内容不预设为(非)错误信息,但广泛包含错误、部分正确和正确的帖子,可能具有或不具有误导性。我们提出并评估了错误信息纠正质量的13个维度,涵盖识别的准确性、解释的事实性以及参考文献的相关性和可信度。结果表明,MUSE能够迅速为社交媒体上的潜在错误信息撰写高质量回应——总体而言,MUSE的性能比GPT-4高出37%,甚至比外行的高质量回应高出29%。这项工作揭示了LLM在有效且高效地帮助对抗现实世界错误信息方面的潜力。