Recent years have witnessed success in AIGC (AI Generated Content). People can make use of a pre-trained diffusion model to generate images of high quality or freely modify existing pictures with only prompts in nature language. More excitingly, the emerging personalization techniques make it feasible to create specific-desired images with only a few images as references. However, this induces severe threats if such advanced techniques are misused by malicious users, such as spreading fake news or defaming individual reputations. Thus, it is necessary to regulate personalization models (i.e., concept censorship) for their development and advancement. In this paper, we focus on the personalization technique dubbed Textual Inversion (TI), which is becoming prevailing for its lightweight nature and excellent performance. TI crafts the word embedding that contains detailed information about a specific object. Users can easily download the word embedding from public websites like Civitai and add it to their own stable diffusion model without fine-tuning for personalization. To achieve the concept censorship of a TI model, we propose leveraging the backdoor technique for good by injecting backdoors into the Textual Inversion embeddings. Briefly, we select some sensitive words as triggers during the training of TI, which will be censored for normal use. In the subsequent generation stage, if the triggers are combined with personalized embeddings as final prompts, the model will output a pre-defined target image rather than images including the desired malicious concept. To demonstrate the effectiveness of our approach, we conduct extensive experiments on Stable Diffusion, a prevailing open-sourced text-to-image model. Our code, data, and results are available at https://concept-censorship.github.io.

翻译：近年来，AIGC（人工智能生成内容）领域取得了显著成功。用户可通过预训练扩散模型，仅凭自然语言提示即可生成高质量图像，或自由修改现有图片。更令人兴奋的是，新兴的个性化技术使得仅需少量参考图像即可创建特定目标图像。然而，此类先进技术一旦被恶意用户滥用（如传播虚假新闻或诽谤他人声誉），将带来严重威胁。因此，有必要对个性化模型实施概念审查，以促进其规范发展。本文聚焦当前因轻量高效而备受青睐的个性化技术——文本反转（Textual Inversion, TI）。该技术通过构建包含特定对象细节信息的词嵌入，使用户可从Civitai等公共平台直接下载此类嵌入，无需微调即可将其集成至个人Stable Diffusion模型中实现个性化生成。为实现文本反转模型的概念审查，我们提出将后门技术用于良性目的：在TI训练过程中，向文本反转嵌入注入后门。具体而言，我们选择若干敏感词汇作为触发器，这些词汇将在正常使用中受到审查。在后续生成阶段，若触发器与个性化嵌入组合为最终提示，模型将输出预定义的目标图像，而非包含恶意概念的目标内容。为验证方法有效性，我们在主流开源文生图模型Stable Diffusion上开展了大量实验。相关代码、数据与结果已公开于https://concept-censorship.github.io。