In recent years, detecting fake multimodal content on social media has drawn increasing attention. Two major forms of deception dominate: human-crafted misinformation (e.g., rumors and misleading posts) and AI-generated content produced by image synthesis models or vision-language models (VLMs). Although both share deceptive intent, they are typically studied in isolation. NLP research focuses on human-written misinformation, while the CV community targets AI-generated artifacts. As a result, existing models are often specialized for only one type of fake content. In real-world scenarios, however, the type of a multimodal post is usually unknown, limiting the effectiveness of such specialized systems. To bridge this gap, we construct the Omnibus Dataset for Multimodal News Deception (OmniFake), a comprehensive benchmark of 127K samples that integrates human-curated misinformation from existing resources with newly synthesized AI-generated examples. Based on this dataset, we propose Unified Multimodal Fake Content Detection (UMFDet), a framework designed to handle both forms of deception. UMFDet leverages a VLM backbone augmented with a Category-aware Mixture-of-Experts (MoE) Adapter to capture category-specific cues, and an attribution chain-of-thought mechanism that provides implicit reasoning guidance for locating salient deceptive signals. Extensive experiments demonstrate that UMFDet achieves robust and consistent performance across both misinformation types, outperforming specialized baselines and offering a practical solution for real-world multimodal deception detection.
翻译:近年来,社交媒体上的多模态虚假内容检测日益受到关注。当前主要的欺骗形式包括人工编造的虚假信息(如谣言和误导性帖文)以及由图像合成模型或视觉语言模型(VLM)生成的AI合成内容。尽管两者均具有欺骗意图,现有研究通常将其割裂处理:自然语言处理领域聚焦于人工撰写的虚假信息,而计算机视觉领域则主要针对AI生成痕迹。这导致现有模型往往仅适用于单一类型的虚假内容。然而在实际场景中,多模态帖文的类型通常未知,限制了此类专用系统的有效性。为弥合这一差距,我们构建了多模态新闻欺骗综合数据集(OmniFake),该基准包含12.7万样本,整合了现有资源中人工标注的虚假信息与新合成的AI生成示例。基于此数据集,我们提出统一多模态虚假内容检测框架(UMFDet),该框架通过增强视觉语言模型主干网络,引入类别感知专家混合适配器以捕捉类别特定特征,并采用属性链式思维机制为定位显著欺骗信号提供隐式推理指导。大量实验表明,UMFDet在两种虚假信息类型上均取得稳健且一致的性能表现,优于专用基线方法,为现实世界的多模态欺骗检测提供了实用解决方案。