Diffusion models are usually evaluated by their final outputs, gradually denoising random noise into meaningful images. Yet, generation unfolds along a trajectory, and analyzing this dynamic process is crucial for understanding how controllable, reliable, and predictable these models are in terms of their success/failure modes. In this work, we ask the question: when does noise turn into a specific concept (e.g., age) and lock in the denoising trajectory? We propose PCI (Prompt-Conditioned Intervention) to study this question. PCI is a training-free and model-agnostic framework for analyzing concept dynamics through diffusion time. The central idea is the analysis of Concept Insertion Success (CIS), defined as the probability that a concept inserted at a given timestep is preserved and reflected in the final image, offering a way to characterize the temporal dynamics of concept formation. Applied to several state-of-the-art text-to-image diffusion models and a broad taxonomy of concepts, PCI reveals diverse temporal behaviors across diffusion models, in which certain phases of the trajectory are more favorable to specific concepts even within the same concept type. These findings also provide actionable insights for text-driven image editing, highlighting when interventions are most effective without requiring access to model internals or training, and yielding quantitatively stronger edits that achieve a balance of semantic accuracy and content preservation than strong baselines. Code is available at: https://adagorgun.github.io/PCI-Project/
翻译:扩散模型通常通过其最终输出来评估,即逐步将随机噪声去噪为有意义的图像。然而,生成过程沿着一条轨迹展开,分析这一动态过程对于理解这些模型在成功/失败模式方面的可控性、可靠性和可预测性至关重要。在本研究中,我们提出以下问题:噪声何时转变为特定概念(例如年龄)并锁定去噪轨迹?我们提出PCI(提示条件干预)来研究这一问题。PCI是一种无需训练且与模型无关的框架,用于通过扩散时间分析概念动态。其核心思想是分析概念插入成功率(CIS),定义为在给定时间步插入的概念在最终图像中被保留和反映的概率,这提供了一种表征概念形成时间动态的方法。将PCI应用于多种先进的文本到图像扩散模型和广泛的概念分类体系,揭示了不同扩散模型间多样的时间行为,其中轨迹的某些阶段对特定概念更为有利,即使在同一概念类型内也是如此。这些发现还为文本驱动的图像编辑提供了可操作的见解,突出了无需访问模型内部或进行训练时干预最有效的时机,并且相比强基线方法,实现了语义准确性与内容保持平衡的定量更强的编辑效果。代码可在以下网址获取:https://adagorgun.github.io/PCI-Project/