360{\deg} omnidirectional images have gained research attention due to their immersive and interactive experience, particularly in AR/VR applications. However, they suffer from lower angular resolution due to being captured by fisheye lenses with the same sensor size for capturing planar images. To solve the above issues, we propose a two-stage framework for 360{\deg} omnidirectional image superresolution. The first stage employs two branches: model A, which incorporates omnidirectional position-aware deformable blocks (OPDB) and Fourier upsampling, and model B, which adds a spatial frequency fusion module (SFF) to model A. Model A aims to enhance the feature extraction ability of 360{\deg} image positional information, while Model B further focuses on the high-frequency information of 360{\deg} images. The second stage performs same-resolution enhancement based on the structure of model A with a pixel unshuffle operation. In addition, we collected data from YouTube to improve the fitting ability of the transformer, and created pseudo low-resolution images using a degradation network. Our proposed method achieves superior performance and wins the NTIRE 2023 challenge of 360{\deg} omnidirectional image super-resolution.

翻译：360{\deg}全向图像因其沉浸式和交互式体验在AR/VR应用中受到研究关注，但由于使用与平面图像相同传感器尺寸的鱼眼镜头捕获，其角分辨率较低。为解决上述问题，我们提出了一种面向360{\deg}全向图像超分辨率的两阶段框架。第一阶段采用两个分支：模型A集成了全向位置感知可变形块（OPDB）和傅里叶上采样，模型B在模型A基础上增加了空间频率融合模块（SFF）。模型A旨在增强360{\deg}图像位置信息的特征提取能力，而模型B进一步关注360{\deg}图像的高频信息。第二阶段基于模型A的结构，通过像素重组操作实现同分辨率增强。此外，我们从YouTube收集数据以提升Transformer的拟合能力，并利用退化网络生成伪低分辨率图像。所提方法取得了优越性能，赢得了NTIRE 2023 360{\deg}全向图像超分辨率挑战赛。