Diffusion models have enabled high-quality, conditional image editing capabilities. We propose to expand their arsenal, and demonstrate that off-the-shelf diffusion models can be used for a wide range of cross-domain compositing tasks. Among numerous others, these include image blending, object immersion, texture-replacement and even CG2Real translation or stylization. We employ a localized, iterative refinement scheme which infuses the injected objects with contextual information derived from the background scene, and enables control over the degree and types of changes the object may undergo. We conduct a range of qualitative and quantitative comparisons to prior work, and exhibit that our method produces higher quality and realistic results without requiring any annotations or training. Finally, we demonstrate how our method may be used for data augmentation of downstream tasks.
翻译:扩散模型已实现高质量、条件式的图像编辑能力。我们提出扩展其应用范围,并证明现成的扩散模型可用于广泛的跨域合成任务。这些任务包括(但不限于)图像融合、对象沉浸、纹理替换,甚至CG2Real翻译或风格化。我们采用局部迭代细化方案,将注入对象与从背景场景中提取的上下文信息融合,并能够控制对象可能经历的变化程度和类型。我们与先前工作进行了一系列定性和定量比较,表明我们的方法无需任何标注或训练即可生成更高质量和更逼真的结果。最后,我们展示了该方法如何用于下游任务的数据增强。