Recovering clear structures from severely blurry inputs is a challenging problem due to the large movements between the camera and the scene. Although some works apply segmentation maps on human face images for deblurring, they cannot handle natural scenes because objects and degradation are more complex, and inaccurate segmentation maps lead to a loss of details. For general scene deblurring, the feature space of the blurry image and corresponding sharp image under the high-level vision task is closer, which inspires us to rely on other tasks (e.g. classification) to learn a comprehensive prior in severe blur removal cases. We propose a cross-level feature learning strategy based on knowledge distillation to learn the priors, which include global contexts and sharp local structures for recovering potential details. In addition, we propose a semantic prior embedding layer with multi-level aggregation and semantic attention transformation to integrate the priors effectively. We introduce the proposed priors to various models, including the UNet and other mainstream deblurring baselines, leading to better performance on severe blur removal. Extensive experiments on natural image deblurring benchmarks and real-world images, such as GoPro and RealBlur datasets, demonstrate our method's effectiveness and generalization ability.
翻译:从严重模糊的输入中恢复清晰结构是一个极具挑战性的问题,这是由于相机与场景之间存在大幅运动。尽管已有研究在面部图像去模糊中应用分割图,但由于自然场景中的物体与退化更为复杂,且不准确的分割图会导致细节丢失,因此此类方法无法处理自然场景。在通用场景去模糊中,模糊图像与对应清晰图像在高层次视觉任务下的特征空间更为接近,这启发我们依赖其他任务(如分类)来学习严重模糊情况下的全面先验信息。我们提出一种基于知识蒸馏的跨层级特征学习策略来学习这些先验,其包含全局上下文和用于恢复潜在细节的清晰局部结构。此外,我们提出一种具有多层级聚合与语义注意力变换的语义先验嵌入层,以有效整合这些先验信息。我们将所提出的先验信息引入多种模型(包括UNet及其他主流去模糊基线模型),从而在严重模糊去除任务中取得更优性能。在自然图像去模糊基准(如GoPro和RealBlur数据集)及真实世界图像上的大量实验表明,我们的方法具有有效性与泛化能力。