Image harmonization, which involves adjusting the foreground of a composite image to attain a unified visual consistency with the background, can be conceptualized as an image-to-image translation task. Diffusion models have recently promoted the rapid development of image-to-image translation tasks . However, training diffusion models from scratch is computationally intensive. Fine-tuning pre-trained latent diffusion models entails dealing with the reconstruction error induced by the image compression autoencoder, making it unsuitable for image generation tasks that involve pixel-level evaluation metrics. To deal with these issues, in this paper, we first adapt a pre-trained latent diffusion model to the image harmonization task to generate the harmonious but potentially blurry initial images. Then we implement two strategies: utilizing higher-resolution images during inference and incorporating an additional refinement stage, to further enhance the clarity of the initially harmonized images. Extensive experiments on iHarmony4 datasets demonstrate the superiority of our proposed method. The code and model will be made publicly available at https://github.com/nicecv/DiffHarmony .

翻译：图像和谐化（即调整合成图像的前景以实现与背景的统一视觉一致性）可被概念化为一种图像到图像的翻译任务。扩散模型近年来推动了图像到图像翻译任务的快速发展。然而，从头训练扩散模型计算成本高昂。微调预训练的潜在扩散模型需要处理由图像压缩自编码器引入的重建误差，这使得该方法不适用于涉及像素级评估指标的图像生成任务。为解决这些问题，本文首先将预训练的潜在扩散模型适应于图像和谐化任务，以生成和谐但可能模糊的初始图像。随后，我们实施两种策略：在推理过程中使用更高分辨率图像，并引入额外的精炼阶段，以进一步提升初始和谐化图像的清晰度。在iHarmony4数据集上的大量实验证明了所提出方法的优越性。代码和模型将公开发布于https://github.com/nicecv/DiffHarmony。