The social media-fuelled explosion of fake news and misinformation supported by tampered images has led to growth in the development of models and datasets for image manipulation detection. However, existing detection methods mostly treat media objects in isolation, without considering the impact of specific manipulations on viewer perception. Forensic datasets are usually analyzed based on the manipulation operations and corresponding pixel-based masks, but not on the semantics of the manipulation, i.e., type of scene, objects, and viewers' attention to scene content. The semantics of the manipulation play an important role in spreading misinformation through manipulated images. In an attempt to encourage further development of semantic-aware forensic approaches to understand visual misinformation, we propose a framework to analyze the trends of visual and semantic saliency in popular image manipulation datasets and their impact on detection.

翻译：社交媒体助燃的假新闻和基于篡改图像的虚假信息爆炸性增长，推动了图像操控检测模型与数据集的发展。然而，现有检测方法大多孤立地处理媒体对象，未考虑特定操控对观众感知的影响。法医数据集通常根据操控操作及其对应的像素级掩码进行分析，而非基于操控的语义信息——即场景类型、物体及其对观众场景内容注意力的影响。操控语义在通过篡改图像传播虚假信息中扮演着重要角色。为促进理解视觉虚假信息的语义感知法医方法进一步发展，我们提出一个分析框架，旨在探究主流图像操控数据集中视觉与语义显著性趋势及其对检测的影响。