Image inpainting refers to the task of generating a complete, natural image based on a partially revealed reference image. Recently, many research interests have been focused on addressing this problem using fixed diffusion models. These approaches typically directly replace the revealed region of the intermediate or final generated images with that of the reference image or its variants. However, since the unrevealed regions are not directly modified to match the context, it results in incoherence between revealed and unrevealed regions. To address the incoherence problem, a small number of methods introduce a rigorous Bayesian framework, but they tend to introduce mismatches between the generated and the reference images due to the approximation errors in computing the posterior distributions. In this paper, we propose COPAINT, which can coherently inpaint the whole image without introducing mismatches. COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions, but approximates the posterior distribution in a way that allows the errors to gradually drop to zero throughout the denoising steps, thus strongly penalizing any mismatches with the reference image. Our experiments verify that COPAINT can outperform the existing diffusion-based methods under both objective and subjective metrics. The codes are available at https://github.com/UCSB-NLP-Chang/CoPaint/.

翻译：图像修复是指基于部分显露的参考图像生成完整、自然图像的任务。近年来，大量研究致力于利用固定扩散模型解决该问题。这些方法通常直接用参考图像或其变体替换中间或最终生成图像中已显露区域。然而，由于未显露区域未被直接修改以匹配上下文，导致已显露与未显露区域之间存在不连贯性。为解决此不连贯问题，少数方法引入了严格的贝叶斯框架，但因其在后验分布计算中存在近似误差，常导致生成图像与参考图像之间出现不匹配。本文提出COPAINT方法，能在不引入不匹配的情况下对整幅图像进行连贯修复。COPAINT同样采用贝叶斯框架联合修改已显露与未显露区域，但通过独特方式近似后验分布，使误差在去噪步骤中逐渐趋近于零，从而有效惩罚与参考图像的任何不匹配。实验证明，COPAINT在主客观指标上均优于现有基于扩散的方法。代码见https://github.com/UCSB-NLP-Chang/CoPaint/。