Image harmonization aims at adjusting the appearance of the foreground to make it more compatible with the background. Without exploring background illumination and its effects on the foreground elements, existing works are incapable of generating a realistic foreground shading. In this paper, we decompose the image harmonization task into two sub-problems: 1) illumination estimation of the background image and 2) re-rendering of foreground objects under background illumination. Before solving these two sub-problems, we first learn a shading-aware illumination descriptor via a well-designed neural rendering framework, of which the key is a shading bases module that generates multiple shading bases from the foreground image. Then we design a background illumination estimation module to extract the illumination descriptor from the background. Finally, the Shading-aware Illumination Descriptor is used in conjunction with the neural rendering framework (SIDNet) to produce the harmonized foreground image containing a novel harmonized shading. Moreover, we construct a photo-realistic synthetic image harmonization dataset that contains numerous shading variations with image-based lighting. Extensive experiments on both synthetic and real data demonstrate the superiority of the proposed method, especially in dealing with foreground shadings.

翻译：图像和谐化旨在调整前景外观以使其与背景更加协调。现有方法因未探索背景光照及其对前景元素的影响，难以生成真实的前景阴影。本文将图像和谐化任务分解为两个子问题：1）背景图像的光照估计，以及2）前景物体在背景光照下的重新渲染。在解决这两个子问题前，我们首先通过精心设计的神经渲染框架学习阴影感知光照描述子，其关键在于一个阴影基元模块，该模块从前景图像生成多个阴影基元。随后设计背景光照估计模块以提取背景光照描述子。最终，将阴影感知光照描述子与神经渲染框架（SIDNet）结合，生成包含新颖和谐化阴影的前景图像。此外，本文构建了一个包含基于图像光照的多种阴影变化的真实感合成图像和谐化数据集。在合成与真实数据上的大量实验表明，所提方法在应对前景阴影方面具有显著优势。