Text-to-image diffusion models are nothing but a revolution, allowing anyone, even without design skills, to create realistic images from simple text inputs. With powerful personalization tools like DreamBooth, they can generate images of a specific person just by learning from his/her few reference images. However, when misused, such a powerful and convenient tool can produce fake news or disturbing content targeting any individual victim, posing a severe negative social impact. In this paper, we explore a defense system called Anti-DreamBooth against such malicious use of DreamBooth. The system aims to add subtle noise perturbation to each user's image before publishing in order to disrupt the generation quality of any DreamBooth model trained on these perturbed images. We investigate a wide range of algorithms for perturbation optimization and extensively evaluate them on two facial datasets over various text-to-image model versions. Despite the complicated formulation of DreamBooth and Diffusion-based text-to-image models, our methods effectively defend users from the malicious use of those models. Their effectiveness withstands even adverse conditions, such as model or prompt/term mismatching between training and testing. Our code will be available at https://github.com/VinAIResearch/Anti-DreamBooth.git.

翻译：文本到图像扩散模型是一场革命，使任何人都能（即使没有设计技能）通过简单的文本输入创建逼真的图像。借助DreamBooth等强大的个性化工具，仅需学习用户的几张参考图像，即可生成该特定人物的图像。然而，此类强大且便捷的工具若被滥用，可能针对任何个人受害者生成虚假新闻或令人不安的内容，从而造成严重的社会负面影响。本文探索了一种名为Anti-DreamBooth的防御系统，旨在对抗DreamBooth的恶意使用。该系统通过向每位用户的图像添加微小的噪声扰动，来破坏基于这些扰动图像训练的任何DreamBooth模型的生成质量。我们研究了多种用于扰动优化的算法，并在两个面部数据集上针对不同版本的文本到图像模型进行了广泛评估。尽管DreamBooth及基于扩散的文本到图像模型公式复杂，我们的方法仍能有效保护用户免受这些模型的恶意利用。甚至在模型或提示/术语在训练和测试间不匹配等不利条件下，其有效性依然稳固。我们的代码将在https://github.com/VinAIResearch/Anti-DreamBooth.git公开。