Deepfakes pose significant security and privacy threats through malicious facial manipulations. While robust watermarking can aid in authenticity verification and source tracking, existing methods often lack the sufficient robustness against Deepfake manipulations. Diffusion models have demonstrated remarkable performance in image generation, enabling the seamless fusion of watermark with image during generation. In this study, we propose a novel robust watermarking framework based on diffusion model, called DiffMark. By modifying the training and sampling scheme, we take the facial image and watermark as conditions to guide the diffusion model to progressively denoise and generate corresponding watermarked image. In the construction of facial condition, we weight the facial image by a timestep-dependent factor that gradually reduces the guidance intensity with the decrease of noise, thus better adapting to the sampling process of diffusion model. To achieve the fusion of watermark condition, we introduce a cross information fusion (CIF) module that leverages a learnable embedding table to adaptively extract watermark features and integrates them with image features via cross-attention. To enhance the robustness of the watermark against Deepfake manipulations, we integrate a frozen autoencoder during training phase to simulate Deepfake manipulations. Additionally, we introduce Deepfake-resistant guidance that employs specific Deepfake model to adversarially guide the diffusion sampling process to generate more robust watermarked images. Experimental results demonstrate the effectiveness of the proposed DiffMark on typical Deepfakes. Our code will be available at https://github.com/vpsg-research/DiffMark.

翻译：深度伪造技术通过恶意面部操纵带来了严重的安全与隐私威胁。尽管鲁棒水印技术有助于真实性验证和来源追踪，但现有方法往往缺乏对抗深度伪造操作的足够鲁棒性。扩散模型在图像生成方面表现出卓越性能，能够在生成过程中实现水印与图像的无缝融合。本研究提出一种基于扩散模型的新型鲁棒水印框架，称为DiffMark。通过修改训练与采样方案，我们将面部图像和水印作为条件，引导扩散模型逐步去噪并生成相应的含水印图像。在面部条件构建中，我们采用时间步相关因子对面部图像进行加权，该因子随噪声减少而逐步降低引导强度，从而更好地适应扩散模型的采样过程。为实现水印条件的融合，我们引入了交叉信息融合模块，该模块利用可学习的嵌入表自适应提取水印特征，并通过交叉注意力机制将其与图像特征进行整合。为增强水印对抗深度伪造操作的鲁棒性，我们在训练阶段集成冻结自编码器以模拟深度伪造操作。此外，我们提出抗深度伪造引导机制，利用特定深度伪造模型对抗性地引导扩散采样过程，以生成更具鲁棒性的含水印图像。实验结果验证了所提DiffMark方法在典型深度伪造场景下的有效性。我们的代码将在https://github.com/vpsg-research/DiffMark 公开。