Natural image stitching (NIS) aims to create one natural-looking mosaic from two overlapping images that capture the same 3D scene from different viewing positions. Challenges inevitably arise when the scene is non-planar and the camera baseline is wide, since parallax becomes not negligible in such cases. In this paper, we propose a novel NIS method using depth maps, which generates natural-looking mosaics against parallax in both overlapping and non-overlapping regions. Firstly, we construct a robust fitting method to filter out the outliers in feature matches and estimate the epipolar geometry between input images. Then, we draw a triangulation of the target image and estimate multiple local homographies, one per triangle, based on the locations of their vertices, the rectified depth values and the epipolar geometry. Finally, the warping image is rendered by the backward mapping of piece-wise homographies. Panorama is then produced via average blending and image inpainting. Experimental results demonstrate that the proposed method not only provides accurate alignment in the overlapping regions but also virtual naturalness in the non-overlapping region.

翻译：自然图像拼接（NIS）旨在从两幅重叠图像中生成一幅自然无缝的拼接图，这两幅图像从不同视角拍摄同一三维场景。当场景非平面且相机基线较宽时，由于视差不可忽略，这一任务面临不可避免的挑战。本文提出一种新颖的基于深度图的NIS方法，可在重叠区域和非重叠区域生成抗视差的自然拼接图。首先，构建鲁棒拟合方法过滤特征匹配中的异常点，并估计输入图像间的极线几何；其次，基于目标图像的三角剖分，结合顶点位置、校正后的深度值及极线几何，为每个三角形估计局部单应性矩阵；最后，通过逐片单应性的反向映射渲染变形图像，并采用平均融合与图像修补生成全景图。实验结果表明，该方法不仅在重叠区域实现精确对齐，还在非重叠区域呈现虚拟自然效果。