We investigate the problem of automatically placing an object into a background image for image compositing. Given a background image and a segmented object, the goal is to train a model to predict plausible placements (location and scale) of the object for compositing. The quality of the composite image highly depends on the predicted location/scale. Existing works either generate candidate bounding boxes or apply sliding-window search using global representations from background and object images, which fail to model local information in background images. However, local clues in background images are important to determine the compatibility of placing the objects with certain locations/scales. In this paper, we propose to learn the correlation between object features and all local background features with a transformer module so that detailed information can be provided on all possible location/scale configurations. A sparse contrastive loss is further proposed to train our model with sparse supervision. Our new formulation generates a 3D heatmap indicating the plausibility of all location/scale combinations in one network forward pass, which is over 10 times faster than the previous sliding-window method. It also supports interactive search when users provide a pre-defined location or scale. The proposed method can be trained with explicit annotation or in a self-supervised manner using an off-the-shelf inpainting model, and it outperforms state-of-the-art methods significantly. The user study shows that the trained model generalizes well to real-world images with diverse challenging scenes and object categories.

翻译：[translated abstract in Chinese] 我们研究图像合成中自动将物体放置到背景图像中的问题。给定背景图像和分割后的物体，目标是训练一个模型来预测物体合成的合理放置（位置和尺度）。合成图像的质量高度依赖于预测的位置/尺度。现有方法要么生成候选边界框，要么使用背景和物体图像的全局表示应用滑动窗口搜索，这无法建模背景图像中的局部信息。然而，背景图像中的局部线索对于确定物体在特定位置/尺度下的放置兼容性至关重要。本文提出使用Transformer模块学习物体特征与所有局部背景特征之间的相关性，从而为所有可能的位置/尺度配置提供详细信息。进一步提出稀疏对比损失，以稀疏监督方式训练模型。我们的新公式在单次网络前向传播中生成一个三维热力图，指示所有位置/尺度组合的合理性，其速度比先前的滑动窗口方法快10倍以上。当用户提供预定义的位置或尺度时，它还支持交互式搜索。所提出的方法可以使用显式标注或利用现成的图像修复模型以自监督方式训练，并且显著优于最先进的方法。用户研究表明，训练后的模型能很好地泛化到具有多样化挑战性场景和物体类别的真实世界图像。