Dual-lens super-resolution (SR) is a practical scenario for reference (Ref) based SR by utilizing the telephoto image (Ref) to assist the super-resolution of the low-resolution wide-angle image (LR input). Different from general RefSR, the Ref in dual-lens SR only covers the overlapped field of view (FoV) area. However, current dual-lens SR methods rarely utilize these specific characteristics and directly perform dense matching between the LR input and Ref. Due to the resolution gap between LR and Ref, the matching may miss the best-matched candidate and destroy the consistent structures in the overlapped FoV area. Different from them, we propose to first align the Ref with the center region (namely the overlapped FoV area) of the LR input by combining global warping and local warping to make the aligned Ref be sharp and consistent. Then, we formulate the aligned Ref and LR center as value-key pairs, and the corner region of the LR is formulated as queries. In this way, we propose a kernel-free matching strategy by matching between the LR-corner (query) and LR-center (key) regions, and the corresponding aligned Ref (value) can be warped to the corner region of the target. Our kernel-free matching strategy avoids the resolution gap between LR and Ref, which makes our network have better generalization ability. In addition, we construct a DuSR-Real dataset with (LR, Ref, HR) triples, where the LR and HR are well aligned. Experiments on three datasets demonstrate that our method outperforms the second-best method by a large margin. Our code and dataset are available at https://github.com/Craigie-Hill/KeDuSR.
翻译:双镜头超分辨率(SR)是利用长焦图像(参考图)辅助低分辨率广角图像(低分辨率输入)超分辨率的实用参考(Ref)超分辨率场景。与通用参考超分辨率不同,双镜头SR中的参考图仅覆盖重叠视场(FoV)区域。然而,现有双镜头SR方法鲜少利用这一特性,直接对低分辨率输入与参考图进行密集匹配。由于低分辨率与参考图之间的分辨率差异,匹配可能遗漏最佳候选点,并破坏重叠视场区域的一致结构。针对此问题,我们提出先通过全局形变与局部形变联合对齐参考图与低分辨率输入的中心区域(即重叠视场区域),使对齐后的参考图同时具备清晰性与一致性。进而将对齐后的参考图与低分辨率中心区域构建为值-键对,将低分辨率角区域构建为查询。基于此,我们提出无核匹配策略,通过低分辨率角(查询)与低分辨率中心(键)区域的匹配,将对应对齐后的参考图(值)形变至目标角区域。该无核匹配策略规避了低分辨率与参考图之间的分辨率差异,使网络具有更优泛化能力。此外,我们构建了包含(低分辨率、参考图、高分辨率)三元组的DuSR-Real数据集,其中低分辨率与高分辨率图像已精确对齐。在三个数据集上的实验表明,本方法以显著优势超越次优方法。代码与数据集公开于https://github.com/Craigie-Hill/KeDuSR。