Image restoration (IR) has been an indispensable and challenging task in the low-level vision field, which strives to improve the subjective quality of images distorted by various forms of degradation. Recently, the diffusion model has achieved significant advancements in the visual generation of AIGC, thereby raising an intuitive question, "whether diffusion model can boost image restoration". To answer this, some pioneering studies attempt to integrate diffusion models into the image restoration task, resulting in superior performances than previous GAN-based methods. Despite that, a comprehensive and enlightening survey on diffusion model-based image restoration remains scarce. In this paper, we are the first to present a comprehensive review of recent diffusion model-based methods on image restoration, encompassing the learning paradigm, conditional strategy, framework design, modeling strategy, and evaluation. Concretely, we first introduce the background of the diffusion model briefly and then present two prevalent workflows that exploit diffusion models in image restoration. Subsequently, we classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR, intending to inspire future development. To evaluate existing methods thoroughly, we summarize the commonly-used dataset, implementation details, and evaluation metrics. Additionally, we present the objective comparison for open-sourced methods across three tasks, including image super-resolution, deblurring, and inpainting. Ultimately, informed by the limitations in existing works, we propose five potential and challenging directions for the future research of diffusion model-based IR, including sampling efficiency, model compression, distortion simulation and estimation, distortion invariant learning, and framework design.
翻译:图像恢复(IR)一直是低层视觉领域中不可或缺且富有挑战性的任务,旨在提升因各种降质形式而失真的图像的主观质量。近年来,扩散模型在AIGC的视觉生成领域取得了显著进展,由此引发一个直观问题:“扩散模型能否提升图像恢复质量?”为回答该问题,一些开创性研究尝试将扩散模型整合到图像恢复任务中,其性能优于先前基于GAN的方法。尽管如此,针对基于扩散模型的图像恢复,目前仍缺乏全面且具启发性的综述。本文首次对近期基于扩散模型的图像恢复方法进行了全面综述,涵盖学习范式、条件策略、框架设计、建模策略及评估方面。具体而言,我们首先简要介绍扩散模型的背景,随后阐述利用扩散模型进行图像恢复的两种主流流程。接着,我们对采用扩散模型用于标准图像恢复及盲/真实世界图像恢复的创新设计进行分类并着重讨论,旨在启发未来发展。为全面评估现有方法,我们总结了常用数据集、实现细节及评估指标。此外,我们对三项目标任务(包括图像超分辨率、去模糊及修复)中的开源方法进行了客观比较。最终,基于现有工作的局限性,我们提出了扩散模型图像恢复未来研究的五个潜在且具挑战性的方向,包括采样效率、模型压缩、失真模拟与估计、失真不变学习及框架设计。