Image completion techniques have made significant progress in filling missing regions (i.e., holes) in images. However, large-hole completion remains challenging due to limited structural information. In this paper, we address this problem by integrating explicit structural guidance into diffusion-based image completion, forming our structure-guided diffusion model (SGDM). It consists of two cascaded diffusion probabilistic models: structure and texture generators. The structure generator generates an edge image representing plausible structures within the holes, which is then used for guiding the texture generation process. To train both generators jointly, we devise a novel strategy that leverages optimal Bayesian denoising, which denoises the output of the structure generator in a single step and thus allows backpropagation. Our diffusion-based approach enables a diversity of plausible completions, while the editable edges allow for editing parts of an image. Our experiments on natural scene (Places) and face (CelebA-HQ) datasets demonstrate that our method achieves a superior or comparable visual quality compared to state-of-the-art approaches. The code is available for research purposes at https://github.com/UdonDa/Structure_Guided_Diffusion_Model.

翻译：图像补全技术在填充图像中的缺失区域（即空洞）方面取得了显著进展。然而，由于结构信息有限，大空洞补全仍然具有挑战性。在本文中，我们通过将显式结构引导整合到基于扩散的图像补全中，形成我们的结构引导扩散模型（SGDM），来解决这一问题。该模型由两个级联的扩散概率模型组成：结构生成器和纹理生成器。结构生成器生成一个边缘图像，该图像表示空洞内合理的结构，随后用于引导纹理生成过程。为了联合训练这两个生成器，我们提出了一种新颖的策略，该策略利用最优贝叶斯去噪，在单步中去除结构生成器输出的噪声，从而允许反向传播。我们的基于扩散的方法能够实现多样化的合理补全结果，同时可编辑的边缘允许对图像部分进行编辑。我们在自然场景（Places）和人脸（CelebA-HQ）数据集上的实验表明，与最先进的方法相比，我们的方法实现了更好或相当的视觉质量。代码可在 https://github.com/UdonDa/Structure_Guided_Diffusion_Model 获取，用于研究目的。