Despite recent advances in inversion-based editing, text-guided image manipulation remains challenging for diffusion models. The primary bottlenecks include 1) the time-consuming nature of the inversion process; 2) the struggle to balance consistency with accuracy; 3) the lack of compatibility with efficient consistency sampling methods used in consistency models. To address the above issues, we start by asking ourselves if the inversion process can be eliminated for editing. We show that when the initial sample is known, a special variance schedule reduces the denoising step to the same form as the multi-step consistency sampling. We name this Denoising Diffusion Consistent Model (DDCM), and note that it implies a virtual inversion strategy without explicit inversion in sampling. We further unify the attention control mechanisms in a tuning-free framework for text-guided editing. Combining them, we present inversion-free editing (InfEdit), which allows for consistent and faithful editing for both rigid and non-rigid semantic changes, catering to intricate modifications without compromising on the image's integrity and explicit inversion. Through extensive experiments, InfEdit shows strong performance in various editing tasks and also maintains a seamless workflow (less than 3 seconds on one single A40), demonstrating the potential for real-time applications. Project Page: https://sled-group.github.io/InfEdit/
翻译:尽管基于反演的编辑技术近期取得了进展,但文本引导的图像编辑对扩散模型而言仍具挑战性。主要瓶颈包括:1)反演过程耗时较长;2)一致性与准确性难以平衡;3)与一致性模型中使用的高效一致性采样方法缺乏兼容性。为解决上述问题,我们首先思考是否可以在编辑中消除反演过程。研究表明,当初始样本已知时,特殊的方差调度可将去噪步骤简化为与多步一致性采样相同的形式。我们将此方法命名为去噪扩散一致性模型(DDCM),并指出其在采样过程中无需显式反演即能实现虚拟反演策略。我们进一步将注意力控制机制统一到一个无需微调的文本引导编辑框架中。结合上述技术,我们提出了无反演编辑方法(InfEdit),该方法可实现刚性与非刚性语义变化的一致且保真的编辑,在保持图像完整性的同时完成复杂修改,且无需显式反演。大量实验表明,InfEdit在各类编辑任务中均表现强劲,同时保持流畅的工作流程(单张A40上耗时不足3秒),展现出实时应用的潜力。项目页面:https://sled-group.github.io/InfEdit/