Recent advances in diffusion models enable many powerful instruments for image editing. One of these instruments is text-driven image manipulations: editing semantic attributes of an image according to the provided text description. % Popular text-conditional diffusion models offer various high-quality image manipulation methods for a broad range of text prompts. Existing diffusion-based methods already achieve high-quality image manipulations for a broad range of text prompts. However, in practice, these methods require high computation costs even with a high-end GPU. This greatly limits potential real-world applications of diffusion-based image editing, especially when running on user devices. In this paper, we address efficiency of the recent text-driven editing methods based on unconditional diffusion models and develop a novel algorithm that learns image manipulations 4.5-10 times faster and applies them 8 times faster. We carefully evaluate the visual quality and expressiveness of our approach on multiple datasets using human annotators. Our experiments demonstrate that our algorithm achieves the quality of much more expensive methods. Finally, we show that our approach can adapt the pretrained model to the user-specified image and text description on the fly just for 4 seconds. In this setting, we notice that more compact unconditional diffusion models can be considered as a rational alternative to the popular text-conditional counterparts.

翻译：近年来，扩散模型的进展催生了多种强大的图像编辑工具。其中一种工具是文本驱动的图像操控：根据给定的文本描述编辑图像的语义属性。现有的基于扩散的方法已能针对广泛的文本提示实现高质量的图像操控。然而，实际应用中，即使使用高端GPU，这些方法也需要高昂的计算成本。这极大地限制了基于扩散的图像编辑在真实场景中的潜在应用，尤其是在用户设备上运行时。本文聚焦于基于无条件扩散模型的近期文本驱动编辑方法的效率问题，并开发了一种新算法，该算法学习图像操控的速度提升4.5-10倍，应用速度提升8倍。我们通过人工标注者在多个数据集上精心评估了本方法的视觉质量与表现力。实验表明，本算法所达到的质量与更昂贵的方法相当。最后，我们展示了本方法能够在仅需4秒的情况下，实时地将预训练模型适配到用户指定的图像和文本描述中。在此设定下，我们注意到更紧凑的无条件扩散模型可被视为流行的文本条件模型的一种合理替代方案。