Multi-task image restoration has gained significant interest due to its inherent versatility and efficiency compared to its single-task counterpart. Despite its potential, performance degradation is observed with an increase in the number of tasks, primarily attributed to the distinct nature of each restoration task. Addressing this challenge, we introduce \mbox{\textbf{DINO-IR}}, a novel multi-task image restoration approach leveraging robust features extracted from DINOv2. Our empirical analysis shows that while shallow features of DINOv2 capture rich low-level image characteristics, the deep features ensure a robust semantic representation insensitive to degradations while preserving high-frequency contour details. Building on these features, we devise specialized components, including multi-layer semantic fusion module, DINO-Restore adaption and fusion module, and DINO perception contrastive loss, to integrate DINOv2 features into the restoration paradigm. Equipped with the aforementioned components, our DINO-IR performs favorably against existing multi-task image restoration approaches in various tasks by a large margin, indicating the superiority and necessity of reinforcing the robust features for multi-task image restoration.
翻译:多任务图像修复因其固有的通用性和相对于单任务方法的高效性而受到广泛关注。尽管潜力巨大,但随着任务数量的增加,模型性能会出现下降,这主要归因于每个修复任务各自的独特性。针对这一挑战,我们提出了\textbf{DINO-IR},一种利用从DINOv2中提取的鲁棒特征的新型多任务图像修复方法。我们的实证分析表明,DINOv2的浅层特征能够捕捉丰富的底层图像特性,而深层特征则在保持高频轮廓细节的同时,确保了对退化不敏感的鲁棒语义表征。基于这些特征,我们设计了专门的组件,包括多层语义融合模块、DINO-Restore适应与融合模块以及DINO感知对比损失,将DINOv2特征集成到修复范式中。借助上述组件,我们的DINO-IR在多项任务中显著优于现有其他多任务图像修复方法,证明了增强鲁棒特征对多任务图像修复的优越性与必要性。