The field of Fake Image Detection and Localization (FIDL) is highly fragmented, encompassing four domains: deepfake detection (Deepfake), image manipulation detection and localization (IMDL), artificial intelligence-generated image detection (AIGC), and document image manipulation localization (Doc). Although individual benchmarks exist in some domains, a unified benchmark for all domains in FIDL remains blank. The absence of a unified benchmark results in significant domain silos, where each domain independently constructs its datasets, models, and evaluation protocols without interoperability, preventing cross-domain comparisons and hindering the development of the entire FIDL field. To close the domain silo barrier, we propose ForensicHub, the first unified benchmark & codebase for all-domain fake image detection and localization. Considering drastic variations on dataset, model, and evaluation configurations across all domains, as well as the scarcity of open-sourced baseline models and the lack of individual benchmarks in some domains, ForensicHub: i) proposes a modular and configuration-driven architecture that decomposes forensic pipelines into interchangeable components across datasets, transforms, models, and evaluators, allowing flexible composition across all domains; ii) fully implements 10 baseline models, 6 backbones, 2 new benchmarks for AIGC and Doc, and integrates 2 existing benchmarks of DeepfakeBench and IMDLBenCo through an adapter-based design; iii) conducts indepth analysis based on the ForensicHub, offering 8 key actionable insights into FIDL model architecture, dataset characteristics, and evaluation standards. ForensicHub represents a significant leap forward in breaking the domain silos in the FIDL field and inspiring future breakthroughs.
翻译:伪造图像检测与定位(FIDL)领域高度碎片化,涵盖四大领域:深度伪造检测(Deepfake)、图像篡改检测与定位(IMDL)、人工智能生成图像检测(AIGC)以及文档图像篡改定位(Doc)。尽管部分领域已存在独立基准,但针对FIDL全领域的统一基准仍属空白。统一基准的缺失导致了严重的领域壁垒,各领域独立构建其数据集、模型与评估协议而缺乏互操作性,阻碍了跨领域比较并制约了整个FIDL领域的发展。为打破领域壁垒,我们提出了ForensicHub——首个面向全领域伪造图像检测与定位的统一基准与代码库。考虑到各领域在数据集、模型及评估配置上的巨大差异,以及开源基线模型的稀缺性和部分领域独立基准的缺失,ForensicHub实现了以下创新:i)提出模块化、配置驱动的架构,将取证流程解耦为跨数据集、变换模块、模型与评估器的可互换组件,支持跨所有领域的灵活组合;ii)完整实现了10个基线模型与6种骨干网络,新建了AIGC与Doc领域的2项基准,并通过基于适配器的设计整合了DeepfakeBench与IMDLBenCo两项现有基准;iii)基于ForensicHub开展深度分析,针对FIDL模型架构、数据集特性及评估标准提出了8项关键可行见解。ForensicHub标志着在打破FIDL领域壁垒、启发未来突破方面取得了重大进展。