Diffusion models have been widely deployed in various image generation tasks, demonstrating an extraordinary connection between image and text modalities. However, they face challenges of being maliciously exploited to generate harmful or sensitive images by appending a specific suffix to the original prompt. Existing works mainly focus on using single-modal information to conduct attacks, which fails to utilize multi-modal features and results in less than satisfactory performance. Integrating multi-modal priors (MMP), i.e. both text and image features, we propose a targeted attack method named MMP-Attack in this work. Specifically, the goal of MMP-Attack is to add a target object into the image content while simultaneously removing the original object. The MMP-Attack shows a notable advantage over existing works with superior universality and transferability, which can effectively attack commercial text-to-image (T2I) models such as DALL-E 3. To the best of our knowledge, this marks the first successful attempt of transfer-based attack to commercial T2I models. Our code is publicly available at \url{https://github.com/ydc123/MMP-Attack}.

翻译：扩散模型已被广泛应用于各类图像生成任务，展现出图像与文本模态间的非凡关联性。然而，这些模型面临被恶意利用的风险——攻击者可通过在原始提示词后附加特定后缀来生成有害或敏感图像。现有研究主要利用单模态信息实施攻击，未能充分利用多模态特征，导致攻击效果欠佳。本文通过整合多模态先验（MMP），即文本与图像特征，提出名为MMP-Attack的定向攻击方法。具体而言，MMP-Attack的目标是在图像内容中添加目标物体的同时移除原始物体。相较于现有方法，MMP-Attack展现出显著的普适性与迁移性优势，可有效攻击DALL-E 3等商业文本到图像（T2I）模型。据我们所知，这是首个成功实现基于迁移攻击的商业T2I模型攻击方案。我们的代码已开源至\url{https://github.com/ydc123/MMP-Attack}。