Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature. In this work, we present our BoundaryDiffusion method for efficient, effective and light-weight semantic control with frozen pre-trained DDMs, without learning any extra networks. As one of the first learning-free diffusion editing works, we start by seeking a comprehensive understanding of the intermediate high-dimensional latent spaces by theoretically and empirically analyzing their probabilistic and geometric behaviors in the Markov chain. We then propose to further explore the critical step for editing in the denoising trajectory that characterizes the convergence of a pre-trained DDM and introduce an automatic search method. Last but not least, in contrast to the conventional understanding that DDMs have relatively poor semantic behaviors, we prove that the critical latent space we found already exhibits semantic subspace boundaries at the generic level in unconditional DDMs, which allows us to do controllable manipulation by guiding the denoising trajectory towards the targeted boundary via a single-step operation. We conduct extensive experiments on multiple DPMs architectures (DDPM, iDDPM) and datasets (CelebA, CelebA-HQ, LSUN-church, LSUN-bedroom, AFHQ-dog) with different resolutions (64, 256), achieving superior or state-of-the-art performance in various task scenarios (image semantic editing, text-based editing, unconditional semantic control) to demonstrate the effectiveness.

翻译：应用预训练的生成去噪扩散模型（DDMs）执行图像语义编辑等下游任务时，现有文献通常需要对DDMs进行微调或学习辅助编辑网络。本文提出BoundaryDiffusion方法，在不学习任何附加网络的情况下，利用冻结的预训练DDMs实现高效、有效且轻量级的语义控制。作为首批无学习扩散编辑工作之一，我们首先通过理论和实证分析马尔可夫链中高维潜空间的概率与几何行为，深入理解其特性；随后针对去噪轨迹中决定预训练DDM收敛的关键编辑步骤，提出自动搜索方法。最后，与DDMs语义表现较弱的传统认知相反，我们证明在无条件DDMs中，所发现的关键潜空间已呈现通用层面的语义子空间边界，这使得通过单步操作引导去噪轨迹指向目标边界即可实现可控操作。我们在多种DPMs架构（DDPM、iDDPM）和数据集（CelebA、CelebA-HQ、LSUN-church、LSUN-bedroom、AFHQ-dog）上，针对不同分辨率（64、256）开展大量实验，在图像语义编辑、文本驱动编辑、无条件语义控制等多种任务场景中取得领先或最优性能，验证了方法的有效性。