Image deblurring is a critical task in the field of image restoration, aiming to eliminate blurring artifacts. However, the challenge of addressing non-uniform blurring leads to an ill-posed problem, which limits the generalization performance of existing deblurring models. To solve the problem, we propose a framework SAM-Deblur, integrating prior knowledge from the Segment Anything Model (SAM) into the deblurring task for the first time. In particular, SAM-Deblur is divided into three stages. First, We preprocess the blurred images, obtain image masks via SAM, and propose a mask dropout method for training to enhance model robustness. Then, to fully leverage the structural priors generated by SAM, we propose a Mask Average Pooling (MAP) unit specifically designed to average SAM-generated segmented areas, serving as a plug-and-play component which can be seamlessly integrated into existing deblurring networks. Finally, we feed the fused features generated by the MAP Unit into the deblurring model to obtain a sharp image. Experimental results on the RealBlurJ, ReloBlur, and REDS datasets reveal that incorporating our methods improves NAFNet's PSNR by 0.05, 0.96, and 7.03, respectively. Code will be available at \href{https://github.com/HPLQAQ/SAM-Deblur}{SAM-Deblur}.
翻译:图像去模糊是图像复原领域的核心任务,旨在消除模糊伪影。然而,非均匀模糊的修复具有病态性,限制了现有去模糊模型的泛化性能。为解决该问题,我们首次提出SAM-Deblur框架,将分割一切模型(SAM)的先验知识融入去模糊任务。具体而言,SAM-Deblur包含三个阶段:首先对模糊图像进行预处理,通过SAM获取图像掩码,并提出用于训练的掩码丢弃方法以增强模型鲁棒性;然后为充分挖掘SAM生成的结构先验,我们设计了一种掩码平均池化单元,专门用于平均SAM生成的分割区域,该单元作为即插即用组件可无缝集成至现有去模糊网络;最后将MAP单元生成的融合特征输入去模糊模型以获得清晰图像。在RealBlurJ、ReloBlur和REDS数据集上的实验结果表明,集成我们的方法后,NAFNet的PSNR分别提升0.05、0.96和7.03。代码将发布于\href{https://github.com/HPLQAQ/SAM-Deblur}{SAM-Deblur}。