The wide-angle lens shows appealing applications in VR technologies, but it introduces severe radial distortion into its captured image. To recover the realistic scene, previous works devote to rectifying the content of the wide-angle image. However, such a rectification solution inevitably distorts the image boundary, which changes related geometric distributions and misleads the current vision perception models. In this work, we explore constructing a win-win representation on both content and boundary by contributing a new learning model, i.e., Rectangling Rectification Network (RecRecNet). In particular, we propose a thin-plate spline (TPS) module to formulate the non-linear and non-rigid transformation for rectangling images. By learning the control points on the rectified image, our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. To relieve the complexity of structure approximation, we then inspire our RecRecNet to learn the gradual deformation rules with a DoF (Degree of Freedom)-based curriculum learning. By increasing the DoF in each curriculum stage, namely, from similarity transformation (4-DoF) to homography transformation (8-DoF), the network is capable of investigating more detailed deformations, offering fast convergence on the final rectangling task. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations. The code and dataset are available at https://github.com/KangLiao929/RecRecNet.

翻译：广角镜头在虚拟现实技术中展现出极具吸引力的应用前景，但其捕获的图像会引入严重的径向畸变。为恢复真实场景，先前的研究致力于校正广角图像的内容。然而，这种校正方案不可避免地会导致图像边界变形，从而改变相关的几何分布并误导当前的视觉感知模型。本文通过提出一种新的学习模型——矩形化校正网络（RecRecNet），探索在内容与边界上构建双赢的表示。具体而言，我们提出了一种薄板样条（TPS）模块，用于对矩形化图像进行非线性和非刚性变换。通过在校正图像上学习控制点，我们的模型能够灵活地将源结构扭曲到目标域，并实现端到端的无监督变形。为减轻结构逼近的复杂性，我们进一步启发RecRecNet采用基于自由度（DoF）的课程学习来学习渐进变形规则。通过在每个课程阶段增加自由度，即从相似变换（4-DoF）到单应变换（8-DoF），网络能够探索更精细的变形，从而在最终矩形化任务上实现快速收敛。实验表明，我们的解决方案在定量与定性评估上均优于对比方法。代码与数据集已开源在 https://github.com/KangLiao929/RecRecNet。