The prevalent use of commercial and open-source diffusion models (DMs) for text-to-image generation prompts risk mitigation to prevent undesired behaviors. Existing concept erasing methods in academia are all based on full parameter or specification-based fine-tuning, from which we observe the following issues: 1) Generation alternation towards erosion: Parameter drift during target elimination causes alternations and potential deformations across all generations, even eroding other concepts at varying degrees, which is more evident with multi-concept erased; 2) Transfer inability & deployment inefficiency: Previous model-specific erasure impedes the flexible combination of concepts and the training-free transfer towards other models, resulting in linear cost growth as the deployment scenarios increase. To achieve non-invasive, precise, customizable, and transferable elimination, we ground our erasing framework on one-dimensional adapters to erase multiple concepts from most DMs at once across versatile erasing applications. The concept-SemiPermeable structure is injected as a Membrane (SPM) into any DM to learn targeted erasing, and meantime the alteration and erosion phenomenon is effectively mitigated via a novel Latent Anchoring fine-tuning strategy. Once obtained, SPMs can be flexibly combined and plug-and-play for other DMs without specific re-tuning, enabling timely and efficient adaptation to diverse scenarios. During generation, our Facilitated Transport mechanism dynamically regulates the permeability of each SPM to respond to different input prompts, further minimizing the impact on other concepts. Quantitative and qualitative results across ~40 concepts, 7 DMs and 4 erasing applications have demonstrated the superior erasing of SPM. Our code and pre-tuned SPMs will be available on the project page https://lyumengyao.github.io/projects/spm.

翻译：商业和开源扩散模型（DMs）在文本到图像生成中的广泛应用，促使需要采取风险缓解措施以防止不当行为。现有学术界的概念擦除方法均基于全参数或特定规范的微调，我们从中发现以下问题：1）生成向侵蚀方向偏移：目标消除过程中的参数漂移导致所有生成结果发生偏移和潜在变形，甚至会不同程度地侵蚀其他概念，在多概念擦除时尤为明显；2）迁移不可行与部署低效：以往针对特定模型的擦除方式阻碍了概念的灵活组合以及向其他模型的免训练迁移，导致部署场景增加时成本线性增长。为实现非侵入式、精准、可定制且可迁移的消除，我们将擦除框架建立在基于一维适配器的基础上，以一次性从多数扩散模型中擦除多个概念，并涵盖多种擦除应用。我们将概念-半渗透结构作为膜（SPM）注入任意扩散模型以学习目标擦除，同时通过新颖的潜空间锚定微调策略有效缓解了偏移和侵蚀现象。获取SPM后，它们可灵活组合并即插即用于其他扩散模型，无需重新微调，从而实现对不同场景的及时高效适配。在生成过程中，我们的Facilitated Transport机制动态调控各SPM的渗透性以响应不同输入提示，进一步减少对其他概念的影响。在约40个概念、7种扩散模型及4类擦除应用上的定量与定性结果证明了SPM的卓越擦除性能。我们的代码与预调优SPM将在项目页面https://lyumengyao.github.io/projects/spm上提供。