In recent years, diffusion models have achieved remarkable success in the realm of high-quality image generation, garnering increased attention. This surge in interest is paralleled by a growing concern over the security threats associated with diffusion models, largely attributed to their susceptibility to malicious exploitation. Notably, recent research has brought to light the vulnerability of diffusion models to backdoor attacks, enabling the generation of specific target images through corresponding triggers. However, prevailing backdoor attack methods rely on manually crafted trigger generation functions, often manifesting as discernible patterns incorporated into input noise, thus rendering them susceptible to human detection. In this paper, we present an innovative and versatile optimization framework designed to acquire invisible triggers, enhancing the stealthiness and resilience of inserted backdoors. Our proposed framework is applicable to both unconditional and conditional diffusion models, and notably, we are the pioneers in demonstrating the backdooring of diffusion models within the context of text-guided image editing and inpainting pipelines. Moreover, we also show that the backdoors in the conditional generation can be directly applied to model watermarking for model ownership verification, which further boosts the significance of the proposed framework. Extensive experiments on various commonly used samplers and datasets verify the efficacy and stealthiness of the proposed framework. Our code is publicly available at https://github.com/invisibleTriggerDiffusion/invisible_triggers_for_diffusion.

翻译：近年来，扩散模型在高质量图像生成领域取得了显著成功，获得了越来越多的关注。与此同时，人们对扩散模型相关的安全威胁也日益担忧，这主要归因于其易受恶意利用的特性。值得注意的是，近期研究揭示了扩散模型对后门攻击的脆弱性，使得攻击者能够通过相应的触发器生成特定的目标图像。然而，现有的后门攻击方法依赖于手动设计的触发器生成函数，通常表现为输入噪声中可被识别的模式，因此容易被人眼检测。本文提出了一种创新且通用的优化框架，旨在获取隐形触发器，从而增强植入后门的隐蔽性和鲁棒性。我们提出的框架适用于无条件与条件扩散模型，并且值得注意的是，我们是首个在文本引导图像编辑与修复流程中演示扩散模型后门植入的研究。此外，我们还证明了条件生成中的后门可直接应用于模型水印技术，以实现模型所有权验证，这进一步提升了所提框架的重要性。在多种常用采样器和数据集上进行的大量实验验证了所提框架的有效性与隐蔽性。我们的代码已在 https://github.com/invisibleTriggerDiffusion/invisible_triggers_for_diffusion 公开。