Dual-lens super-resolution (SR) is a practical scenario for reference (Ref) based SR by utilizing the telephoto image (Ref) to assist the super-resolution of the low-resolution wide-angle image (LR input). Different from general RefSR, the Ref in dual-lens SR only covers the overlapped field of view (FoV) area. However, current dual-lens SR methods rarely utilize these specific characteristics and directly perform dense matching between the LR input and Ref. Due to the resolution gap between LR and Ref, the matching may miss the best-matched candidate and destroy the consistent structures in the overlapped FoV area. Different from them, we propose to first align the Ref with the center region (namely the overlapped FoV area) of the LR input by combining global warping and local warping to make the aligned Ref be sharp and consistent. Then, we formulate the aligned Ref and LR center as value-key pairs, and the corner region of the LR is formulated as queries. In this way, we propose a kernel-free matching strategy by matching between the LR-corner (query) and LR-center (key) regions, and the corresponding aligned Ref (value) can be warped to the corner region of the target. Our kernel-free matching strategy avoids the resolution gap between LR and Ref, which makes our network have better generalization ability. In addition, we construct a DuSR-Real dataset with (LR, Ref, HR) triples, where the LR and HR are well aligned. Experiments on three datasets demonstrate that our method outperforms the second-best method by a large margin. Our code and dataset are available at https://github.com/ZifanCui/KeDuSR.
翻译:双镜头超分辨率(SR)是通过利用长焦图像(参考图)辅助低分辨率广角图像(低分辨率输入)进行超分辨率的实用参考(Ref)SR场景。与常规RefSR不同,双镜头SR中的参考图仅覆盖重叠视场(FoV)区域。然而,现有双镜头SR方法鲜少利用这一特性,而是直接对低分辨率输入与参考图进行密集匹配。由于低分辨率与参考图之间存在分辨率差异,这种匹配可能错失最佳匹配候选,并破坏重叠视场区域的一致性结构。区别于现有方法,我们提出通过联合全局变形与局部变形,将参考图对齐至低分辨率输入的中心区域(即重叠视场区域),使对齐后的参考图既清晰又结构一致。随后,我们将对齐后的参考图与低分辨率中心区域构建为值-键对,并将低分辨率角点区域构建为查询。在此基础上,我们提出无核匹配策略,通过低分辨率角点(查询)与低分辨率中心(键)区域间的匹配,将对应的对齐参考图(值)变形至目标图像的角点区域。该策略避免了低分辨率与参考图之间的分辨率差异问题,使网络具备更强的泛化能力。此外,我们构建了包含(低分辨率、参考图、高分辨率)三元组的DuSR-Real数据集,其中低分辨率与高分辨率图像已精确对齐。在三个数据集上的实验表明,本方法以显著优势超越当前最优方法。相关代码与数据集已开源至https://github.com/ZifanCui/KeDuSR。