Recent years have witnessed success in AIGC (AI Generated Content). People can make use of a pre-trained diffusion model to generate images of high quality or freely modify existing pictures with only prompts in nature language. More excitingly, the emerging personalization techniques make it feasible to create specific-desired images with only a few images as references. However, this induces severe threats if such advanced techniques are misused by malicious users, such as spreading fake news or defaming individual reputations. Thus, it is necessary to regulate personalization models (i.e., concept censorship) for their development and advancement. In this paper, we focus on the personalization technique dubbed Textual Inversion (TI), which is becoming prevailing for its lightweight nature and excellent performance. TI crafts the word embedding that contains detailed information about a specific object. Users can easily download the word embedding from public websites like Civitai and add it to their own stable diffusion model without fine-tuning for personalization. To achieve the concept censorship of a TI model, we propose leveraging the backdoor technique for good by injecting backdoors into the Textual Inversion embeddings. Briefly, we select some sensitive words as triggers during the training of TI, which will be censored for normal use. In the subsequent generation stage, if the triggers are combined with personalized embeddings as final prompts, the model will output a pre-defined target image rather than images including the desired malicious concept. To demonstrate the effectiveness of our approach, we conduct extensive experiments on Stable Diffusion, a prevailing open-sourced text-to-image model. Our code, data, and results are available at https://concept-censorship.github.io.

翻译：[译摘要] 近年来，AIGC（人工智能生成内容）取得了显著成功。用户仅需通过自然语言提示词，即可利用预训练扩散模型生成高质量图像，或对现有图片进行自由修改。更令人振奋的是，新兴的个性化技术使得仅凭少量参考图像即可生成特定需求的图像成为可能。然而，若此类先进技术被恶意用户滥用（例如传播虚假信息或诽谤他人名誉），将引发严重威胁。因此，有必要对个性化模型实施监管（即概念审查）以促进其健康发展。本文聚焦于文本反演（TI）这一个性化技术——该技术凭借轻量化特性与优异性能正日益普及。文本反演可生成包含特定对象细节信息的词嵌入，用户可直接从Civitai等公共网站下载该嵌入，无需微调即可将其集成至自有稳定扩散模型实现个性化生成。为实现TI模型的概念审查，我们提出通过向文本反演嵌入中注入后门来善用后门技术：具体而言，在TI训练阶段选取敏感词汇作为触发器，这些词汇在常规使用中将受到审查。在后续生成阶段，若将触发器与个性化嵌入组合为最终提示词输入，模型将输出预设目标图像，而非包含恶意概念的目标图像。为验证方法的有效性，我们在开源文生图主流模型Stable Diffusion上开展了大量实验。相关代码、数据及结果已开源至https://concept-censorship.github.io。