Diffusion Models (DMs) are state-of-the-art generative models that learn a reversible corruption process from iterative noise addition and denoising. They are the backbone of many generative AI applications, such as text-to-image conditional generation. However, recent studies have shown that basic unconditional DMs (e.g., DDPM and DDIM) are vulnerable to backdoor injection, a type of output manipulation attack triggered by a maliciously embedded pattern at model input. This paper presents a unified backdoor attack framework (VillanDiffusion) to expand the current scope of backdoor analysis for DMs. Our framework covers mainstream unconditional and conditional DMs (denoising-based and score-based) and various training-free samplers for holistic evaluations. Experiments show that our unified framework facilitates the backdoor analysis of different DM configurations and provides new insights into caption-based backdoor attacks on DMs. Our code is available on GitHub: \url{https://github.com/IBM/villandiffusion}

翻译：扩散模型（DMs）是最先进的生成模型，通过迭代噪声添加与去噪学习可逆的损坏过程。它们构成了文本到图像条件生成等众多生成式AI应用的核心。然而，近期研究表明，基础的无条件扩散模型（如DDPM和DDIM）易受后门注入攻击——一种通过在模型输入中嵌入恶意模式来操纵输出的攻击类型。本文提出统一的后门攻击框架（VillanDiffusion），旨在扩展现有扩散模型后门分析的范围。我们的框架涵盖了主流无条件与条件扩散模型（基于去噪和基于分数的模型），以及多种免训练采样器，以实现全面评估。实验表明，该统一框架促进了不同扩散模型配置的后门分析，并为基于标题的后门攻击提供了新见解。我们的代码已在GitHub上开源：\url{https://github.com/IBM/villandiffusion}