The generalization ability of deepfake detectors is vital for their applications in real-world scenarios. One effective solution to enhance this ability is to train the models with manually-blended data, which we termed "blendfake", encouraging models to learn generic forgery artifacts like blending boundary. Interestingly, current SoTA methods utilize blendfake without incorporating any deepfake data in their training process. This is likely because previous empirical observations suggest that vanilla hybrid training (VHT), which combines deepfake and blendfake data, results in inferior performance to methods using only blendfake data (so-called "1+1<2"). Therefore, a critical question arises: Can we leave deepfake behind and rely solely on blendfake data to train an effective deepfake detector? Intuitively, as deepfakes also contain additional informative forgery clues (e.g., deep generative artifacts), excluding all deepfake data in training deepfake detectors seems counter-intuitive. In this paper, we rethink the role of blendfake in detecting deepfakes and formulate the process from "real to blendfake to deepfake" to be a progressive transition. Specifically, blendfake and deepfake can be explicitly delineated as the oriented pivot anchors between "real-to-fake" transitions. The accumulation of forgery information should be oriented and progressively increasing during this transition process. To this end, we propose an Oriented Progressive Regularizor (OPR) to establish the constraints that compel the distribution of anchors to be discretely arranged. Furthermore, we introduce feature bridging to facilitate the smooth transition between adjacent anchors. Extensive experiments confirm that our design allows leveraging forgery information from both blendfake and deepfake effectively and comprehensively.
翻译:深度伪造检测器的泛化能力对其在现实场景中的应用至关重要。一种提升该能力的有效方案是使用人工混合数据(我们称之为“混合伪造”)训练模型,以促使模型学习通用的伪造痕迹(如混合边界)。有趣的是,当前最先进的方法在训练过程中仅使用混合伪造数据,而未纳入任何深度伪造数据。这很可能是因为先前的实证观察表明,结合深度伪造与混合伪造数据的传统混合训练方法,其性能反而不如仅使用混合伪造数据的方法(即所谓的“1+1<2”)。因此,一个关键问题随之产生:我们能否舍弃深度伪造数据,仅依赖混合伪造数据来训练有效的深度伪造检测器?直觉上,由于深度伪造数据也包含额外的信息性伪造线索(例如深度生成痕迹),在训练深度伪造检测器时完全排除深度伪造数据似乎有违常理。本文重新思考了混合伪造在检测深度伪造中的作用,并将“从真实到混合伪造再到深度伪造”的过程形式化为一种渐进式过渡。具体而言,混合伪造与深度伪造可被明确界定为“真实到伪造”过渡过程中的定向枢轴锚点。在此过渡过程中,伪造信息的积累应具有方向性并逐步增加。为此,我们提出了一种定向渐进正则化器,以建立约束条件,迫使锚点的分布呈离散排列。此外,我们引入了特征桥接技术,以促进相邻锚点之间的平滑过渡。大量实验证实,我们的设计能够有效且全面地利用来自混合伪造与深度伪造的伪造信息。