The generative AI technology offers an increasing variety of tools for generating entirely synthetic images that are increasingly indistinguishable from real ones. Unlike methods that alter portions of an image, the creation of completely synthetic images presents a unique challenge and several Synthetic Image Detection (SID) methods have recently appeared to tackle it. Yet, there is often a large gap between experimental results on benchmark datasets and the performance of methods in the wild. To better address the evaluation needs of SID and help close this gap, this paper introduces a benchmarking framework that integrates several state-of-the-art SID models. Our selection of integrated models was based on the utilization of varied input features, and different network architectures, aiming to encompass a broad spectrum of techniques. The framework leverages recent datasets with a diverse set of generative models, high level of photo-realism and resolution, reflecting the rapid improvements in image synthesis technology. Additionally, the framework enables the study of how image transformations, common in assets shared online, such as JPEG compression, affect detection performance. SIDBench is available on https://github.com/mever-team/sidbench and is designed in a modular manner to enable easy inclusion of new datasets and SID models.

翻译：生成式AI技术提供了越来越多生成完全合成图像的工具，这些图像与真实图像的区分难度日益增加。与修改图像部分区域的方法不同，完全合成图像的生成带来了独特的挑战，近期涌现出多种合成图像检测（Synthetic Image Detection, SID）方法。然而，基准数据集上的实验结果与真实场景下方法性能之间往往存在显著差距。为更好地满足SID评估需求并弥合这一差距，本文提出一个集成多种前沿SID模型的基准测试框架。我们基于输入特征的多样性和网络架构的差异性选择集成模型，旨在涵盖广泛的技术手段。该框架采用包含多种生成模型、高真实感与高分辨率的近期数据集，反映图像合成技术的快速进步。此外，框架支持研究在线共享资产中常见的图像变换（如JPEG压缩）对检测性能的影响。SIDBench代码开源于https://github.com/mever-team/sidbench，采用模块化设计便于新数据集和SID模型的灵活集成。