In this paper, we propose LF-PGVIO, a Visual-Inertial-Odometry (VIO) framework for large Field-of-View (FoV) cameras with a negative plane using points and geodesic segments. Notoriously, when the FoV of a panoramic camera reaches the negative half-plane, the image cannot be unfolded into a single pinhole image. Moreover, if a traditional straight-line detection method is directly applied to the original panoramic image, it cannot be normally used due to the large distortions in the panoramas and remains under-explored in the literature. To address these challenges, we put forward LF-PGVIO, which can provide line constraints for cameras with large FoV, even for cameras with negative-plane FoV, and directly extract omnidirectional curve segments from the raw omnidirectional image. We propose an Omnidirectional Curve Segment Detection (OCSD) method combined with a camera model which is applicable to images with large distortions, such as panoramic annular images, fisheye images, and various panoramic images. Each point on the image is projected onto the sphere, and the detected omnidirectional curve segments in the image named geodesic segments must satisfy the criterion of being a geodesic segment on the unit sphere. The detected geodesic segment is sliced into multiple straight-line segments according to the radian of the geodesic, and descriptors are extracted separately and recombined to obtain new descriptors. Based on descriptor matching, we obtain the constraint relationship of the 3D line segments between multiple frames. In our VIO system, we use sliding window optimization using point feature residuals, line feature residuals, and IMU residuals. Our evaluation of the proposed system on public datasets demonstrates that LF-PGVIO outperforms state-of-the-art methods in terms of accuracy and robustness. Code will be open-sourced at https://github.com/flysoaryun/LF-PGVIO.

翻译：本文提出LF-PGVIO——一种针对具备负半平面的广角相机（Large Field-of-View, FoV）的视觉惯性里程计（Visual-Inertial-Odometry, VIO）框架，融合点特征与测地线段约束。众所周知，当全景相机的视场覆盖负半平面时，图像无法展开为单针孔图像。此外，若将传统直线检测方法直接应用于原始全景图像，由于全景图像存在显著畸变，该方法无法正常使用，且该问题在现有文献中尚未充分探索。为应对这些挑战，我们提出LF-PGVIO框架，该框架能为大视场相机提供线特征约束，甚至适用于具有负半平面临界的相机，并能直接从原始全向图像中提取全向曲线段。我们提出一种全向曲线段检测（Omnidirectional Curve Segment Detection, OCSD）方法，该方法结合了适用于大畸变图像（如全景环形图像、鱼眼图像及各类全景图像）的相机模型。图像中每个点被投影至球面上，提取出的全向曲线段（命名为测地线段）需满足单位球面上测地线段的标准。检测到的测地线段根据其弧度被分割为多条直线段，分别提取描述子后重组以生成新描述子。基于描述子匹配，我们获得多帧间三维线段的约束关系。在本VIO系统中，我们采用滑动窗口优化，融合点特征残差、线特征残差及IMU残差。在公开数据集上的评估表明，LF-PGVIO在精度与鲁棒性方面均优于现有最优方法。代码将在https://github.com/flysoaryun/LF-PGVIO 开源。