Text-to-image (T2I) models, such as Stable Diffusion, have exhibited remarkable performance in generating high-quality images from text descriptions in recent years. However, text-to-image models may be tricked into generating not-safe-for-work (NSFW) content, particularly in sexual scenarios. Existing countermeasures mostly focus on filtering inappropriate inputs and outputs, or suppressing improper text embeddings, which can block explicit NSFW-related content (e.g., naked or sexy) but may still be vulnerable to adversarial prompts inputs that appear innocent but are ill-intended. In this paper, we present SafeGen, a framework to mitigate unsafe content generation by text-to-image models in a text-agnostic manner. The key idea is to eliminate unsafe visual representations from the model regardless of the text input. In this way, the text-to-image model is resistant to adversarial prompts since unsafe visual representations are obstructed from within. Extensive experiments conducted on four datasets demonstrate SafeGen's effectiveness in mitigating unsafe content generation while preserving the high-fidelity of benign images. SafeGen outperforms eight state-of-the-art baseline methods and achieves 99.1% sexual content removal performance. Furthermore, our constructed benchmark of adversarial prompts provides a basis for future development and evaluation of anti-NSFW-generation methods.

翻译：文本到图像模型（如Stable Diffusion）近年来在根据文本描述生成高质量图像方面展现出卓越性能。然而，该类模型可能被诱导生成不适宜工作场所（NSFW）的内容，尤其是在色情场景中。现有应对措施主要集中于过滤不当输入与输出，或抑制不当文本嵌入，这虽能阻断明确涉及NSFW的内容（如裸体或性感），但面对看似无害实则恶意的对抗性提示词输入时仍可能失效。本文提出SafeGen框架，以文本无关的方式减轻文本到图像模型的不安全内容生成。其核心思想是无论输入何种文本，均从模型中消除不安全视觉表征。通过这种方式，文本到图像模型能够抵抗对抗性提示词，因为不安全视觉表征从模型内部被阻断。在四个数据集上开展的大量实验表明，SafeGen在保持良性图像高保真度的同时，能有效减轻不安全内容生成。该框架优于八种最先进的基线方法，实现了99.1%的色情内容移除性能。此外，我们构建的对抗性提示词基准为未来反NSFW生成方法的发展与评估提供了基础。