Advanced text-to-image models such as DALL-E 2 and Midjourney possess the capacity to generate highly realistic images, raising significant concerns regarding the potential proliferation of unsafe content. This includes adult, violent, or deceptive imagery of political figures. Despite claims of rigorous safety mechanisms implemented in these models to restrict the generation of not-safe-for-work (NSFW) content, we successfully devise and exhibit the first prompt attacks on Midjourney, resulting in the production of abundant photorealistic NSFW images. We reveal the fundamental principles of such prompt attacks and suggest strategically substituting high-risk sections within a suspect prompt to evade closed-source safety measures. Our novel framework, SurrogatePrompt, systematically generates attack prompts, utilizing large language models, image-to-text, and image-to-image modules to automate attack prompt creation at scale. Evaluation results disclose an 88% success rate in bypassing Midjourney's proprietary safety filter with our attack prompts, leading to the generation of counterfeit images depicting political figures in violent scenarios. Both subjective and objective assessments validate that the images generated from our attack prompts present considerable safety hazards.

翻译：先进的文本到图像模型，如DALL-E 2和Midjourney，具备生成高度逼真图像的能力，这引发了人们对不安全内容潜在扩散的严重担忧，包括涉及成人类、暴力类或政治人物欺骗性图像。尽管这些模型声称实施了严格的安全机制以限制不适宜工作场所(NSFW)内容的生成，但我们首次成功设计并展示了针对Midjourney的提示攻击，从而生成大量照片级的NSFW图像。我们揭示了此类提示攻击的基本原理，并提出通过策略性地替换可疑提示中的高风险部分来规避闭源安全措施。我们的新型框架SurrogatePrompt系统性地生成攻击提示，利用大语言模型、图像到文本和图像到图像模块自动化大规模攻击提示的创建。评估结果显示，我们的攻击提示在绕过Midjourney专有安全过滤器方面成功率达88%，进而生成了描述政治人物处于暴力场景的伪造图像。主观与客观评估均证实，由我们攻击提示生成的图像构成了显著的安全隐患。