Subject-driven image generation aims at generating images containing customized subjects, which has recently drawn enormous attention from the research community. However, the previous works cannot precisely control the background and position of the target subject. In this work, we aspire to fill the void and propose two novel subject-driven sub-tasks, i.e., Subject Replacement and Subject Addition. The new tasks are challenging in multiple aspects: replacing a subject with a customized one can change its shape, texture, and color, while adding a target subject to a designated position in a provided scene necessitates a context-aware posture. To conquer these two novel tasks, we first manually curate a new dataset DreamEditBench containing 22 different types of subjects, and 440 source images with different difficulty levels. We plan to host DreamEditBench as a platform and hire trained evaluators for standard human evaluation. We also devise an innovative method DreamEditor to resolve these tasks by performing iterative generation, which enables a smooth adaptation to the customized subject. In this project, we conduct automatic and human evaluations to understand the performance of DreamEditor and baselines on DreamEditBench. For Subject Replacement, we found that the existing models are sensitive to the shape and color of the original subject. The model failure rate will dramatically increase when the source and target subjects are highly different. For Subject Addition, we found that the existing models cannot easily blend the customized subjects into the background smoothly, leading to noticeable artifacts in the generated image. We hope DreamEditBench can become a standard platform to enable future investigations toward building more controllable subject-driven image editing. Our project homepage is https://dreameditbenchteam.github.io/.

翻译：主体驱动图像生成旨在生成包含自定义主体的图像，近年来引起了研究界的广泛关注。然而，先前的工作无法精确控制目标主体的背景和位置。本文旨在填补这一空白，提出两个新颖的主体驱动子任务，即主体替换和主体添加。这些新任务在多个方面具有挑战性：将一个主体替换为自定义主体会改变其形状、纹理和颜色，而将目标主体添加到给定场景中的指定位置则需要考虑上下文的姿态。为攻克这两个新任务，我们首先手动整理了一个新数据集DreamEditBench，包含22种不同类型的主体和440张不同难度等级的源图像。我们计划将DreamEditBench作为一个平台进行托管，并聘请经过训练的评估人员进行标准化人工评估。我们还设计了一种创新方法DreamEditor，通过执行迭代生成来解决这些任务，从而实现对自定义主体的平滑适应。在本项目中，我们进行了自动评估和人工评估，以了解DreamEditor及基线方法在DreamEditBench上的性能。对于主体替换，我们发现现有模型对原始主体的形状和颜色敏感。当源主体和目标主体差异较大时，模型失败率会急剧上升。对于主体添加，我们发现现有模型难以将自定义主体平滑地融入背景中，导致生成图像中出现明显伪影。我们希望DreamEditBench能成为标准平台，推动未来对更可控的主体驱动图像编辑的研究。我们的项目主页为：https://dreameditbenchteam.github.io/。