Reference-based super-resolution (RefSR) has gained considerable success in the field of super-resolution with the addition of high-resolution reference images to reconstruct low-resolution (LR) inputs with more high-frequency details, thereby overcoming some limitations of single image super-resolution (SISR). Previous research in the field of RefSR has mostly focused on two crucial aspects. The first is accurate correspondence matching between the LR and the reference (Ref) image. The second is the effective transfer and aggregation of similar texture information from the Ref images. Nonetheless, an important detail of perceptual loss and adversarial loss has been underestimated, which has a certain adverse effect on texture transfer and reconstruction. In this study, we propose a feature reuse framework that guides the step-by-step texture reconstruction process through different stages, reducing the negative impacts of perceptual and adversarial loss. The feature reuse framework can be used for any RefSR model, and several RefSR approaches have improved their performance after being retrained using our framework. Additionally, we introduce a single image feature embedding module and a texture-adaptive aggregation module. The single image feature embedding module assists in reconstructing the features of the LR inputs itself and effectively lowers the possibility of including irrelevant textures. The texture-adaptive aggregation module dynamically perceives and aggregates texture information between the LR inputs and the Ref images using dynamic filters. This enhances the utilization of the reference texture while reducing reference misuse. The source code is available at https://github.com/Yi-Yang355/FRFSR.
翻译:基于参考图像的超分辨率(RefSR)通过引入高分辨率参考图像来重建低分辨率(LR)输入,以获取更多高频细节,从而克服了单图像超分辨率(SISR)的部分局限性,在该领域取得了显著成功。以往RefSR领域的研究主要关注两个关键方面:一是LR图像与参考(Ref)图像之间的精确对应匹配,二是从参考图像中有效迁移和聚合相似纹理信息。然而,感知损失和对抗损失这一重要细节被低估,对纹理迁移和重建产生了不利影响。本研究提出了一种特征重用框架,该框架通过不同阶段引导逐步纹理重建过程,减少了感知损失和对抗损失的负面影响。该特征重用框架可适用于任何RefSR模型,多项RefSR方法在使用该框架重新训练后性能得到提升。此外,我们还引入了单图像特征嵌入模块和纹理自适应聚合模块。单图像特征嵌入模块有助于重建LR输入本身的特征,有效降低了引入不相关纹理的可能性。纹理自适应聚合模块利用动态滤波器动态感知和聚合LR输入与参考图像之间的纹理信息,在增强参考纹理利用率的同时减少了参考误用。源代码可访问https://github.com/Yi-Yang355/FRFSR获取。