Diffusion customization methods have achieved impressive results with only a minimal number of user-provided images. However, existing approaches customize concepts collectively, whereas real-world applications often require sequential concept integration. This sequential nature can lead to catastrophic forgetting, where previously learned concepts are lost. In this paper, we investigate concept forgetting and concept confusion in the continual customization. To tackle these challenges, we present ConceptGuard, a comprehensive approach that combines shift embedding, concept-binding prompts and memory preservation regularization, supplemented by a priority queue which can adaptively update the importance and occurrence order of different concepts. These strategies can dynamically update, unbind and learn the relationship of the previous concepts, thus alleviating concept forgetting and confusion. Through comprehensive experiments, we show that our approach outperforms all the baseline methods consistently and significantly in both quantitative and qualitative analyses.
翻译:扩散模型定制方法仅需用户提供少量图像即可获得令人印象深刻的效果。然而,现有方法通常对概念进行集体定制,而实际应用往往需要顺序集成多个概念。这种顺序特性可能导致灾难性遗忘,即先前学习的概念被丢失。本文研究了持续定制中的概念遗忘与概念混淆问题。为应对这些挑战,我们提出了ConceptGuard,一种综合性的方法,它结合了偏移嵌入、概念绑定提示和记忆保持正则化,并辅以一个能自适应更新不同概念重要性及出现顺序的优先级队列。这些策略能够动态更新、解绑并学习先前概念之间的关系,从而缓解概念遗忘与混淆。通过全面的实验,我们证明本方法在定量与定性分析中均一致且显著地优于所有基线方法。