With generative models proliferating at a rapid rate, there is a growing need for general purpose fake image detectors. In this work, we first show that the existing paradigm, which consists of training a deep network for real-vs-fake classification, fails to detect fake images from newer breeds of generative models when trained to detect GAN fake images. Upon analysis, we find that the resulting classifier is asymmetrically tuned to detect patterns that make an image fake. The real class becomes a sink class holding anything that is not fake, including generated images from models not accessible during training. Building upon this discovery, we propose to perform real-vs-fake classification without learning; i.e., using a feature space not explicitly trained to distinguish real from fake images. We use nearest neighbor and linear probing as instantiations of this idea. When given access to the feature space of a large pretrained vision-language model, the very simple baseline of nearest neighbor classification has surprisingly good generalization ability in detecting fake images from a wide variety of generative models; e.g., it improves upon the SoTA by +15.07 mAP and +25.90% acc when tested on unseen diffusion and autoregressive models.

翻译：随着生成模型快速发展，对通用伪造图像检测器的需求日益增长。本文首先表明，现有范式（即训练深度网络进行真实与伪造分类）在检测GAN生成图像时，当面对新型生成模型的伪造图像时会失效。通过分析，我们发现所得分类器会非对称地调整以检测使图像伪造的特定模式，导致真实类别成为容纳所有非伪造图像的"汇"类别，包括训练中未出现模型生成的图像。基于这一发现，我们提出无需学习的真实与伪造图像分类方法，即使用未明确训练以区分真伪图像的特征空间。我们通过最近邻分类和线性探测实现该思想。当使用大规模预训练视觉-语言模型的特征空间时，这一极其简单的最近邻分类基准在检测多种生成模型产生的伪造图像时展现出惊人的泛化能力：例如，在未见过的扩散模型和自回归模型测试中，相较现有最佳方法提升了+15.07 mAP和+25.90%的准确率。