Diffusion models have enabled high-quality, conditional image editing capabilities. We propose to expand their arsenal, and demonstrate that off-the-shelf diffusion models can be used for a wide range of cross-domain compositing tasks. Among numerous others, these include image blending, object immersion, texture-replacement and even CG2Real translation or stylization. We employ a localized, iterative refinement scheme which infuses the injected objects with contextual information derived from the background scene, and enables control over the degree and types of changes the object may undergo. We conduct a range of qualitative and quantitative comparisons to prior work, and exhibit that our method produces higher quality and realistic results without requiring any annotations or training. Finally, we demonstrate how our method may be used for data augmentation of downstream tasks.
翻译:扩散模型已实现高质量、可控的图像编辑能力。我们提出扩展其应用范畴,并证明现有扩散模型可广泛适用于各类跨域合成任务,包括但不限于图像融合、物体嵌入、纹理替换乃至CG2Real风格迁移与艺术化处理。我们采用局部迭代优化方案,将注入对象与背景场景的上下文信息深度融合,同时实现对物体变化程度与类型的可控调节。通过与现有方法进行多维度定性与定量对比,我们证明了该方法无需标注数据或额外训练即可生成更高质量且逼真的合成结果。最后,我们展示了该方法在下游任务数据增强中的潜在应用价值。