As a common image editing operation, image composition (object insertion) aims to combine the foreground from one image and another background image, to produce a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency (e.g., incompatible illumination), geometry inconsistency (e.g., unreasonable size), and semantic inconsistency (e.g., mismatched semantic context). The image composition task could be decomposed into multiple sub-tasks, in which each sub-task targets one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow (resp., reflection) generation aims to generate plausible shadow (resp., reflection) for the foreground. These sub-tasks can be executed sequentially or in parallel to acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition. In this paper, we conduct a comprehensive survey over the sub-tasks and combined task of image composition. For each one, we summarize the existing methods, available datasets, and common evaluation metrics. Datasets and codes for image composition are summarized at https://github.com/bcmi/Awesome-Object-Insertion. We have also contributed the first image composition toolbox: libcom https://github.com/bcmi/libcom, which assembles 10+ image-composition-related functions. The ultimate goal of this toolbox is to solve all image composition problems with simple `import libcom'. Based on libcom toolbox, we also develop an online image composition workbench https://libcom.ustcnewly.com.
翻译:作为常见的图像编辑操作,图像合成(目标插入)旨在将前景物体从一幅图像中分离并与另一幅背景图像结合,生成合成图像。然而,众多因素会导致合成图像失真,这些问题可归纳为前景与背景间的三类不一致性:外观不一致(如光照不匹配)、几何不一致(如尺寸不合理)和语义不一致(如背景语义错位)。图像合成任务可分解为多个子任务,每个子任务针对特定问题:目标放置旨在确定前景物体的合理比例、位置与形状;图像融合用于消除前景与背景之间的不自然边界;图像协调调整前景的光照统计特征;阴影(反射)生成技术为前景生成逼真的阴影(反射)效果。这些子任务可顺序执行或并行处理以获得真实感合成图像。据我们所知,目前尚无图像合成领域的系统性综述。本文对图像合成的各个子任务及其联合任务进行了全面综述,针对每个任务归纳了现有方法、可用数据集及通用评估指标。图像合成相关数据集与代码已汇总于https://github.com/bcmi/Awesome-Object-Insertion,同时我们创建了首个图像合成工具库libcom(https://github.com/bcmi/libcom),该库集成了10余项图像合成功能模块,最终目标是通过简单的`import libcom`语句解决所有图像合成问题。基于libcom工具库,我们还开发了在线图像合成工作台https://libcom.ustcnewly.com。