The limitations of task-specific and general image restoration methods for specific degradation have prompted the development of all-in-one image restoration techniques. However, the diversity of patterns among multiple degradation, along with the significant uncertainties in mapping between degraded images of different severities and their corresponding undistorted versions, pose significant challenges to the all-in-one restoration tasks. To address these challenges, we propose Perceive-IR, an all-in-one image restorer designed to achieve fine-grained quality control that enables restored images to more closely resemble their undistorted counterparts, regardless of the type or severity of degradation. Specifically, Perceive-IR contains two stages: (1) prompt learning stage and (2) restoration stage. In the prompt learning stage, we leverage prompt learning to acquire a fine-grained quality perceiver capable of distinguishing three-tier quality levels by constraining the prompt-image similarity in the CLIP perception space. Subsequently, this quality perceiver and difficulty-adaptive perceptual loss are integrated as a quality-aware learning strategy to realize fine-grained quality control in restoration stage. For the restoration stage, a semantic guidance module (SGM) and compact feature extraction (CFE) are proposed to further promote the restoration process by utilizing the robust semantic information from the pre-trained large scale vision models and distinguishing degradation-specific features. Extensive experiments demonstrate that our Perceive-IR outperforms state-of-the-art methods in all-in-one image restoration tasks and exhibit superior generalization ability when dealing with unseen tasks.
翻译:针对特定退化的任务专用与通用图像复原方法存在局限性,这推动了一体化图像复原技术的发展。然而,多种退化模式间的多样性,以及不同严重程度退化图像与其对应未失真版本之间映射关系的显著不确定性,给一体化复原任务带来了重大挑战。为解决这些挑战,我们提出了Perceive-IR,这是一种旨在实现细粒度质量控制的一体化图像复原器,使得复原图像无论面对何种退化类型或严重程度,都能更接近其未失真对应版本。具体而言,Perceive-IR包含两个阶段:(1) 提示学习阶段和(2) 复原阶段。在提示学习阶段,我们利用提示学习,通过在CLIP感知空间中约束提示-图像相似度,获得能够区分三个层级质量水平的细粒度质量感知器。随后,该质量感知器与难度自适应的感知损失被整合为一种质量感知学习策略,以在复原阶段实现细粒度质量控制。对于复原阶段,我们提出了语义引导模块(SGM)和紧凑特征提取(CFE),通过利用预训练大规模视觉模型提供的鲁棒语义信息并区分退化特异性特征,进一步促进复原过程。大量实验表明,我们的Perceive-IR在一体化图像复原任务中优于现有最先进方法,并且在处理未见任务时展现出卓越的泛化能力。