Diffusion models have emerged as powerful priors for image editing tasks such as inpainting and local modification, where the objective is to generate realistic content that remains consistent with observed regions. In particular, zero-shot approaches that leverage a pretrained diffusion model, without any retraining, have been shown to achieve highly effective reconstructions. However, state-of-the-art zero-shot methods typically rely on a sequence of surrogate likelihood functions, whose scores are used as proxies for the ideal score. This procedure however requires vector-Jacobian products through the denoiser at every reverse step, introducing significant memory and runtime overhead. To address this issue, we propose a new likelihood surrogate that yields simple and efficient to sample Gaussian posterior transitions, sidestepping the backpropagation through the denoiser network. Our extensive experiments show that our method achieves strong observation consistency compared with fine-tuned baselines and produces coherent, high-quality reconstructions, all while significantly reducing inference cost. Code is available at https://github.com/YazidJanati/ding.
翻译:扩散模型作为强大的先验知识,已被广泛用于图像修复和局部修改等图像编辑任务,其目标是在已观测区域中生成连贯且真实的内容。特别地,利用预训练扩散模型(无需重新训练)的零样本方法已被证明能够实现高度有效的重建。然而,最先进的零样本方法通常依赖一系列替代似然函数,其得分被用作理想得分的代理。但此过程需要在每个反向步骤中通过去噪器计算向量-雅可比积,这引入了显著的内存和运行时开销。为解决此问题,我们提出一种新的似然替代方法,该方法能生成简洁高效的高斯后验转移采样,从而避免通过去噪器网络进行反向传播。大量实验表明,与微调基线相比,我们的方法在保持强观测一致性的同时,生成了连贯且高质量的重建结果,并显著降低了推理成本。代码可从 https://github.com/YazidJanati/ding 获取。