Diffusion models have significantly improved the performance of image editing. Existing methods realize various approaches to achieve high-quality image editing, including but not limited to text control, dragging operation, and mask-and-inpainting. Among these, instruction-based editing stands out for its convenience and effectiveness in following human instructions across diverse scenarios. However, it still focuses on simple editing operations like adding, replacing, or deleting, and falls short of understanding aspects of world dynamics that convey the realistic dynamic nature in the physical world. Therefore, this work, EditWorld, introduces a new editing task, namely world-instructed image editing, which defines and categorizes the instructions grounded by various world scenarios. We curate a new image editing dataset with world instructions using a set of large pretrained models (e.g., GPT-3.5, Video-LLava and SDXL). To enable sufficient simulation of world dynamics for image editing, our EditWorld trains model in the curated dataset, and improves instruction-following ability with designed post-edit strategy. Extensive experiments demonstrate our method significantly outperforms existing editing methods in this new task. Our dataset and code will be available at https://github.com/YangLing0818/EditWorld

翻译：扩散模型显著提升了图像编辑的性能。现有方法通过多种途径实现高质量图像编辑，包括但不限于文本控制、拖拽操作以及掩码修复。其中，基于指令的编辑因其在多样化场景中遵循人类指令的便捷性与高效性而表现突出。然而，该方法仍局限于添加、替换或删除等简单编辑操作，未能充分理解体现物理世界真实动态特性的世界动态层面。为此，本研究提出EditWorld，引入一种新的编辑任务——世界指令图像编辑，该任务基于多样化世界场景对指令进行定义与分类。我们利用一系列大型预训练模型（如GPT-3.5、Video-LLava与SDXL）构建了包含世界指令的新型图像编辑数据集。为充分模拟图像编辑中的世界动态，EditWorld在构建的数据集上训练模型，并通过设计的后编辑策略提升指令跟随能力。大量实验表明，我们的方法在此新任务中显著优于现有编辑方法。数据集与代码将在https://github.com/YangLing0818/EditWorld 公开。