Large Language Models (LLMs) have garnered significant attention for their powerful ability in natural language understanding and reasoning. In this paper, we present a comprehensive empirical study to explore the performance of LLMs on misinformation detection tasks. This study stands as the pioneering investigation into the understanding capabilities of multiple LLMs regarding both content and propagation across social media platforms. Our empirical studies on five misinformation detection datasets show that LLMs with diverse prompts achieve comparable performance in text-based misinformation detection but exhibit notably constrained capabilities in comprehending propagation structure compared to existing models in propagation-based misinformation detection. Besides, we further design four instruction-tuned strategies to enhance LLMs for both content and propagation-based misinformation detection. These strategies boost LLMs to actively learn effective features from multiple instances or hard instances, and eliminate irrelevant propagation structures, thereby achieving better detection performance. Extensive experiments further demonstrate LLMs would play a better capacity in content and propagation structure under these proposed strategies and achieve promising detection performance. These findings highlight the potential ability of LLMs to detect misinformation.

翻译：摘要：大型语言模型因其在自然语言理解与推理方面的强大能力而备受关注。本文通过一项全面的实证研究，探索了大型语言模型在虚假信息检测任务中的表现。本研究首次系统考察了多种大型语言模型对社交媒体平台上内容与传播结构的理解能力。在五个虚假信息检测数据集上的实证结果表明，采用不同提示策略的大型语言模型在基于文本的虚假信息检测中表现相当，但在理解传播结构方面，相较于现有基于传播的虚假信息检测模型，其能力显著受限。此外，我们进一步设计了四种指令微调策略，以增强大型语言模型在内容与传播双重维度下的虚假信息检测能力。这些策略促使模型主动从多个实例或困难实例中学习有效特征，并滤除无关传播结构，从而获得更优的检测性能。大量实验进一步证明，在这些策略的引导下，大型语言模型在内容与传播结构方面展现出更强的能力，并取得了具有前景的检测效果。这些发现凸显了大型语言模型在虚假信息检测中的潜在能力。