Image restoration aims to reconstruct degraded images, e.g., denoising or deblurring. Existing works focus on designing task-specific methods and there are inadequate attempts at universal methods. However, simply unifying multiple tasks into one universal architecture suffers from uncontrollable and undesired predictions. To address those issues, we explore prompt learning in universal architectures for image restoration tasks. In this paper, we present Degradation-aware Visual Prompts, which encode various types of image degradation, e.g., noise and blur, into unified visual prompts. These degradation-aware prompts provide control over image processing and allow weighted combinations for customized image restoration. We then leverage degradation-aware visual prompts to establish a controllable and universal model for image restoration, called ProRes, which is applicable to an extensive range of image restoration tasks. ProRes leverages the vanilla Vision Transformer (ViT) without any task-specific designs. Furthermore, the pre-trained ProRes can easily adapt to new tasks through efficient prompt tuning with only a few images. Without bells and whistles, ProRes achieves competitive performance compared to task-specific methods and experiments can demonstrate its ability for controllable restoration and adaptation for new tasks. The code and models will be released in \url{https://github.com/leonmakise/ProRes}.
翻译:图像复原旨在重建退化图像,例如去噪或去模糊。现有工作主要聚焦于设计特定任务的方法,而对通用方法的尝试不足。然而,简单地将多个任务统一到一个通用架构中会导致不可控且不理想的预测结果。为解决这些问题,我们在通用架构中探索了针对图像复原任务的提示学习。本文提出了退化感知视觉提示,将多种图像退化类型(如噪声和模糊)编码为统一的视觉提示。这些退化感知提示能够控制图像处理过程,并支持加权组合以实现定制化图像复原。随后,我们利用退化感知视觉提示建立了一个可控且通用的图像复原模型,称为ProRes,适用于广泛的图像复原任务。ProRes采用原生Vision Transformer (ViT)架构,无需任何任务特定设计。此外,预训练的ProRes可以通过高效的提示调优仅凭少量图像轻松适应新任务。无需额外复杂设计,ProRes在性能上与任务特定方法相当,实验证明了其在可控复原和新任务适应方面的能力。代码和模型将发布于\url{https://github.com/leonmakise/ProRes}。