Diffusion Models (DMs) are state-of-the-art generative models that learn a reversible corruption process from iterative noise addition and denoising. They are the backbone of many generative AI applications, such as text-to-image conditional generation. However, recent studies have shown that basic unconditional DMs (e.g., DDPM and DDIM) are vulnerable to backdoor injection, a type of output manipulation attack triggered by a maliciously embedded pattern at model input. This paper presents a unified backdoor attack framework (VillanDiffusion) to expand the current scope of backdoor analysis for DMs. Our framework covers mainstream unconditional and conditional DMs (denoising-based and score-based) and various training-free samplers for holistic evaluations. Experiments show that our unified framework facilitates the backdoor analysis of different DM configurations and provides new insights into caption-based backdoor attacks on DMs. Our code is available on GitHub: \url{https://github.com/IBM/villandiffusion}

翻译：扩散模型（DMs）是最先进的生成模型，它通过迭代加噪和去噪来学习一个可逆的破坏过程。该模型是许多生成式AI应用的支柱，例如文本到图像的条件生成。然而，近期研究表明，基本的无条件扩散模型（如DDPM和DDIM）容易受到后门注入攻击，这是一种通过在模型输入中嵌入恶意模式来操纵输出的攻击。本文提出了一个统一的后门攻击框架（VillanDiffusion），以扩展当前扩散模型后门分析的范围。我们的框架涵盖了主流的无条件扩散模型和条件扩散模型（基于去噪和基于分数的）以及各种免训练采样器，以实现全面评估。实验表明，该统一框架有助于对不同配置的扩散模型进行后门分析，并为基于标题的扩散模型后门攻击提供了新见解。我们的代码已在GitHub上开源：\url{https://github.com/IBM/villandiffusion}