As Deepfake contents continue to proliferate on the internet, advancing face manipulation forensics has become a pressing issue. To combat this emerging threat, previous methods mainly focus on studying how to distinguish authentic and manipulated face images. Despite impressive, image-level classification lacks explainability and is limited to some specific application scenarios. Existing forgery localization methods suffer from imprecise and inconsistent pixel-level annotations. To alleviate these problems, this paper first re-constructs the FaceForensics++ dataset by introducing pixel-level annotations, then builds an extensive benchmark for localizing tampered regions. Next, a novel Multi-Spectral Class Center Network (MSCCNet) is proposed for face manipulation detection and localization. Specifically, inspired by the power of frequency-related forgery traces, we design Multi-Spectral Class Center (MSCC) module to learn more generalizable and semantic-agnostic features. Based on the features of different frequency bands, the MSCC module collects multispectral class centers and computes pixel-to-class relations. Applying multi-spectral class-level representations suppresses the semantic information of the visual concepts, which is insensitive to manipulations. Furthermore, we propose a Multi-level Features Aggregation (MFA) module to employ more low-level forgery artifacts and structure textures. Experimental results quantitatively and qualitatively indicate the effectiveness and superiority of the proposed MSCCNet on comprehensive localization benchmarks. We expect this work to inspire more studies on pixel-level face manipulation localization. The annotations and codes are available.

翻译：随着深度伪造内容在互联网上的持续扩散，推进人脸伪造取证已成为一项紧迫问题。为应对这一新兴威胁，以往方法主要聚焦于研究如何区分真实与伪造的人脸图像。尽管图像级分类效果显著，但其缺乏可解释性，且局限于特定应用场景。现有伪造定位方法受限于不精确且不一致的像素级标注。为解决这些问题，本文首先通过引入像素级标注重构了FaceForensics++数据集，进而为篡改区域定位构建了全面的基准。其次，提出了一种新颖的多光谱类中心网络（MSCCNet），用于人脸伪造检测与定位。具体而言，受频率相关伪造痕迹强大能力的启发，我们设计了多光谱类中心（MSCC）模块，以学习更具泛化性且语义无关的特征。基于不同频带的特征，MSCC模块收集多光谱类中心并计算像素与类之间的关联。应用多光谱类级表示可抑制视觉概念的语义信息，从而使模型对伪造操作不敏感。此外，我们提出多层级特征聚合（MFA）模块，以利用更多的低层伪造伪影和结构纹理。实验结果从定量和定性两个角度表明，所提出的MSCCNet在全面定位基准上具有有效性和优越性。我们期望这项工作能激发更多关于像素级人脸伪造定位的研究。相关标注与代码已公开。