A common problem for composite images is the incompatibility of their foreground and background components. Image harmonization aims to solve this problem, making the whole image look more authentic and coherent. Most existing solutions predict lookup tables (LUTs) or reconstruct images, utilizing various attributes of composite images. Recent approaches have primarily focused on employing global transformations like normalization and color curve rendering to achieve visual consistency, and they often overlook the importance of local visual coherence. We present a patch-based harmonization network consisting of novel Patch-based normalization (PN) blocks and a feature extractor based on statistical color transfer. Extensive experiments demonstrate the network's high generalization capability for different domains. Our network achieves state-of-the-art results on the iHarmony4 dataset. Also, we created a new human portrait harmonization dataset based on FFHQ and checked the proposed method to show the generalization ability by achieving the best metrics on it. The benchmark experiments confirm that the suggested patch-based normalization block and feature extractor effectively improve the network's capability to harmonize portraits. Our code and model baselines are publicly available.
翻译:复合图像中前景与背景成分的不兼容是常见问题。图像和谐化旨在解决这一难题,使整幅图像呈现更真实、更协调的视觉效果。现有方法主要依赖预测查找表或图像重建技术,利用复合图像的多种属性进行优化。近期研究多侧重于采用全局变换手段(如归一化与色彩曲线渲染)实现视觉一致性,却往往忽视局部视觉连贯性的重要性。我们提出一种基于块的和谐化网络,该网络由新型块归一化模块与基于统计颜色传递的特征提取器构成。大量实验证明该网络对不同领域具有高度泛化能力,在iHarmony4数据集上达到了最优水平。此外,我们基于FFHQ数据集构建了全新的人像和谐化数据集,通过在该数据集上获得最佳指标验证了所提方法的泛化性能。基准测试证实,所提出的块归一化模块与特征提取器能够有效提升网络对人像的和谐化处理能力。我们的代码与基线模型已公开提供。