The rapid advancement of generative models has significantly enhanced the quality of AI-generated images, raising concerns about misinformation and the erosion of public trust. Detecting AI-generated images has thus become a critical challenge, particularly in terms of generalizing to unseen generative models. Existing methods using frozen pre-trained CLIP models show promise in generalization but treat the image encoder as a basic feature extractor, failing to fully exploit its potential. In this paper, we perform an in-depth analysis of the frozen CLIP image encoder (CLIP-ViT), revealing that it effectively clusters real images in a high-level, abstract feature space. However, it does not truly possess the ability to distinguish between real and AI-generated images. Based on this analysis, we propose a Masking-based Pre-trained model Fine-Tuning (MPFT) strategy, which introduces a Texture-Aware Masking (TAM) mechanism to mask textured areas containing generative model-specific patterns during fine-tuning. This approach compels CLIP-ViT to attend to the "distributional deviations"from authentic images for AI-generated image detection, thereby achieving enhanced generalization performance. Extensive experiments on the GenImage and UniversalFakeDetect datasets demonstrate that our method, fine-tuned with only a minimal number of images, significantly outperforms existing approaches, achieving up to 98.2% and 94.6% average accuracy on the two datasets, respectively.

翻译：生成模型的快速发展显著提升了AI生成图像的质量，引发了关于虚假信息和公众信任侵蚀的担忧。因此，检测AI生成图像已成为一项关键挑战，特别是在泛化至未见生成模型方面。现有方法使用冻结预训练的CLIP模型在泛化性上展现出潜力，但仅将图像编码器视为基础特征提取器，未能充分发挥其潜力。本文对冻结的CLIP图像编码器（CLIP-ViT）进行了深入分析，发现其在高层次抽象特征空间中能有效聚类真实图像，但并未真正具备区分真实图像与AI生成图像的能力。基于此分析，我们提出了一种基于掩码的预训练模型微调（MPFT）策略，该策略通过引入纹理感知掩码（TAM）机制，在微调过程中对包含生成模型特定模式的纹理区域进行掩码处理。该方法迫使CLIP-ViT关注AI生成图像相对于真实图像的“分布偏差”以实现检测，从而获得增强的泛化性能。在GenImage和UniversalFakeDetect数据集上的大量实验表明，我们的方法仅需极少量图像进行微调，即可显著超越现有方法，在两个数据集上分别达到98.2%和94.6%的平均准确率。