Diffusion-based personalized visual content generation technologies have achieved significant breakthroughs, allowing for the creation of specific objects by just learning from a few reference photos. However, when misused to fabricate fake news or unsettling content targeting individuals, these technologies could cause considerable societal harm. To address this problem, current methods generate adversarial samples by adversarially maximizing the training loss, thereby disrupting the output of any personalized generation model trained with these samples. However, the existing methods fail to achieve effective defense and maintain stealthiness, as they overlook the intrinsic properties of diffusion models. In this paper, we introduce a novel Dual-Domain Anti-Personalization framework (DDAP). Specifically, we have developed Spatial Perturbation Learning (SPL) by exploiting the fixed and perturbation-sensitive nature of the image encoder in personalized generation. Subsequently, we have designed a Frequency Perturbation Learning (FPL) method that utilizes the characteristics of diffusion models in the frequency domain. The SPL disrupts the overall texture of the generated images, while the FPL focuses on image details. By alternating between these two methods, we construct the DDAP framework, effectively harnessing the strengths of both domains. To further enhance the visual quality of the adversarial samples, we design a localization module to accurately capture attentive areas while ensuring the effectiveness of the attack and avoiding unnecessary disturbances in the background. Extensive experiments on facial benchmarks have shown that the proposed DDAP enhances the disruption of personalized generation models while also maintaining high quality in adversarial samples, making it more effective in protecting privacy in practical applications.

翻译：基于扩散的个性化视觉内容生成技术已取得重大突破，仅需从少量参考照片中学习即可生成特定对象。然而，当这些技术被滥用于伪造针对个人的虚假新闻或令人不安的内容时，可能造成严重的社会危害。为解决这一问题，现有方法通过对抗性最大化训练损失来生成对抗样本，从而破坏任何使用这些样本训练的个性化生成模型的输出。然而，现有方法未能实现有效防御并保持隐蔽性，因为它们忽略了扩散模型的内在特性。本文提出了一种新颖的双域反个性化框架（DDAP）。具体而言，我们通过利用个性化生成中图像编码器的固定性和扰动敏感性，开发了空间扰动学习（SPL）方法。随后，我们设计了一种频域扰动学习（FPL）方法，该方法利用扩散模型在频域的特性。SPL破坏生成图像的整体纹理，而FPL专注于图像细节。通过交替使用这两种方法，我们构建了DDAP框架，有效结合了两个域的优势。为进一步提升对抗样本的视觉质量，我们设计了一个定位模块，在确保攻击有效性的同时精确捕捉关注区域，并避免对背景造成不必要的干扰。在面部基准数据集上的大量实验表明，所提出的DDAP增强了对个性化生成模型的破坏效果，同时保持了对抗样本的高质量，使其在实际应用中能更有效地保护隐私。