The goal of image harmonization is adjusting the foreground appearance in a composite image to make the whole image harmonious. To construct paired training images, existing datasets adopt different ways to adjust the illumination statistics of foregrounds of real images to produce synthetic composite images. However, different datasets have considerable domain gap and the performances on small-scale datasets are limited by insufficient training data. In this work, we explore learnable augmentation to enrich the illumination diversity of small-scale datasets for better harmonization performance. In particular, our designed SYthetic COmposite Network (SycoNet) takes in a real image with foreground mask and a random vector to learn suitable color transformation, which is applied to the foreground of this real image to produce a synthetic composite image. Comprehensive experiments demonstrate the effectiveness of our proposed learnable augmentation for image harmonization. The code of SycoNet is released at https://github.com/bcmi/SycoNet-Adaptive-Image-Harmonization.

翻译：图像协调的目标是调整合成图像中的前景外观，使整幅图像和谐统一。为构建成对训练图像，现有数据集采用不同方法调整真实图像前景的照明统计量，以生成合成复合图像。然而，不同数据集存在显著领域差异，且小规模数据集因训练数据不足导致性能受限。本文探索了可学习增强方法，以丰富小规模数据集的照明多样性，从而提升协调性能。具体而言，我们设计的合成复合网络（SycoNet）以真实图像及其前景掩码和随机向量为输入，学习合适的色彩变换，并将其应用于该真实图像的前景区域，生成合成复合图像。综合实验表明，我们提出的可学习增强方法在图像协调中具有显著有效性。SycoNet代码已发布至 https://github.com/bcmi/SycoNet-Adaptive-Image-Harmonization。