Fine-tuning diffusion models through personalized datasets is an acknowledged method for improving generation quality across downstream tasks, which, however, often inadvertently generates unintended concepts such as watermarks and QR codes, attributed to the limitations in image sources and collecting methods within specific downstream tasks. Existing solutions suffer from eliminating these unintentionally learned implicit concepts, primarily due to the dependency on the model's ability to recognize concepts that it actually cannot discern. In this work, we introduce Geom-Erasing, a novel approach that successfully removes the implicit concepts with either an additional accessible classifier or detector model to encode geometric information of these concepts into text domain. Moreover, we propose Implicit Concept, a novel image-text dataset imbued with three implicit concepts (i.e., watermarks, QR codes, and text) for training and evaluation. Experimental results demonstrate that Geom-Erasing not only identifies but also proficiently eradicates implicit concepts, revealing a significant improvement over the existing methods. The integration of geometric information marks a substantial progression in the precise removal of implicit concepts in diffusion models.

翻译：通过个性化数据集微调扩散模型是公认的提升下游任务生成质量的方法，然而受限于特定下游任务的图像来源与采集方式，此类方法常会无意生成水印、QR码等非目标概念。现有解决方案难以消除这些意外习得的隐式概念，根本原因在于其依赖模型对自身无法辨识概念的识别能力。本文提出Geom-Erasing这一创新方法，通过额外可访问的分类器或检测器模型，将隐式概念的几何信息编码至文本域，成功实现此类概念的移除。此外，我们构建了包含水印、QR码、文本三种隐式概念的新型图文数据集Implicit Concept，用于模型训练与评估。实验表明，Geom-Erasing不仅能精准识别隐式概念，更能有效根除这些概念，相较现有方法展现显著优势。几何信息的集成标志着扩散模型中隐式概念精确移除领域的重大进展。