AIGC (AI-Generated Content) has achieved tremendous success in many applications such as text-to-image tasks, where the model can generate high-quality images with diverse prompts, namely, different descriptions in natural languages. More surprisingly, the emerging personalization techniques even succeed in describing unseen concepts with only a few personal images as references, and there have been some commercial platforms for sharing the valuable personalized concept. However, such an advanced technique also introduces a severe threat, where malicious users can misuse the target concept to generate highly-realistic illegal images. Therefore, it becomes necessary for the platform to trace malicious users and hold them accountable. In this paper, we focus on guarding the most popular lightweight personalization model, ie, Textual Inversion (TI). To achieve it, we propose the novel concept watermarking, where watermark information is embedded into the target concept and then extracted from generated images based on the watermarked concept. Specifically, we jointly train a watermark encoder and a watermark decoder with the sampler in the loop. It shows great resilience to different diffusion sampling processes possibly chosen by malicious users, meanwhile preserving utility for normal use. In practice, the concept owner can upload his concept with different watermarks (ie, serial numbers) to the platform, and the platform allocates different users with different serial numbers for subsequent tracing and forensics.
翻译:人工智能生成内容(AIGC)在文本到图像等任务中取得了巨大成功,模型能够根据多样化提示(即自然语言中的不同描述)生成高质量图像。更令人惊讶的是,新兴的个性化技术甚至能够仅通过少量个人图像作为参考,成功描述未见过的概念,目前已有商业平台用于分享这些有价值的个性化概念。然而,这种先进技术也引入了严重威胁——恶意用户可能滥用目标概念生成高度逼真的非法图像。因此,平台有必要追踪恶意用户并追究其责任。本文聚焦于保护最流行的轻量级个性化模型——文本反转(Textual Inversion, TI)。为此,我们提出了一种新颖的概念水印方法,将水印信息嵌入目标概念中,并基于带水印的概念从生成图像中提取水印信息。具体而言,我们联合训练水印编码器与水印解码器,并将采样器纳入循环训练过程。该方法对恶意用户可能选择的不同扩散采样过程具有强大的鲁棒性,同时保留了正常使用的实用性。在实际应用中,概念所有者可将带有不同水印(即序列号)的概念上传至平台,平台为不同用户分配不同序列号,以便后续追踪与取证。