Deepfakes have recently raised significant trust issues and security concerns among the public. Compared to CNN face forgery detectors, ViT-based methods take advantage of the expressivity of transformers, achieving superior detection performance. However, these approaches still exhibit the following limitations: (1). Fully fine-tuning ViT-based models from ImageNet weights demands substantial computational and storage resources; (2). ViT-based methods struggle to capture local forgery clues, leading to model bias and limited generalizability. To tackle these challenges, this work introduces Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized yet parameter-efficient ViT-based approach. MoE-FFD only updates lightweight Low-Rank Adaptation (LoRA) and Adapter layers while keeping the ViT backbone frozen, thereby achieving parameter-efficient training. Moreover, MoE-FFD leverages the expressivity of transformers and local priors of CNNs to simultaneously extract global and local forgery clues. Additionally, novel MoE modules are designed to scale the model's capacity and select optimal forgery experts, further enhancing forgery detection performance. The proposed MoE learning scheme can be seamlessly adapted to various transformer backbones in a plug-and-play manner. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art face forgery detection performance with reduced parameter overhead. The code will be released upon acceptance.

翻译：深度伪造技术近年来引发了公众显著的信任危机与安全担忧。相较于基于CNN的人脸伪造检测器，基于ViT（视觉Transformer）的方法借助Transformer的表达能力，取得了更优的检测性能。然而，这些方法仍存在以下局限：（1）从ImageNet权重对ViT模型进行全参数微调需要大量的计算和存储资源；（2）基于ViT的方法难以捕获局部伪造线索，导致模型偏差与泛化能力受限。为应对这些挑战，本文提出了一种面向人脸伪造检测的混合专家模块（MoE-FFD），这是一种兼具通用性与参数高效性的ViT方法。MoE-FFD仅更新轻量级的低秩适配（LoRA）与适配器层，同时冻结ViT骨干网络，从而实现参数高效训练。此外，MoE-FFD结合Transformer的表达能力与CNN的局部先验，同步提取全局与局部伪造线索。同时，本文设计了新型MoE模块以扩展模型容量并选择最优伪造专家，进一步提升了伪造检测性能。所提出的MoE学习方案可即插即用地无缝适配多种Transformer骨干网络。大量实验结果表明，该方法在降低参数开销的同时实现了最先进的人脸伪造检测性能。相关代码将在论文接收后公开。