A Colored point cloud, as a simple and efficient 3D representation, has many advantages in various fields, including robotic navigation and scene reconstruction. This representation is now commonly used in 3D reconstruction tasks relying on cameras and LiDARs. However, fusing data from these two types of sensors is poorly performed in many existing frameworks, leading to unsatisfactory mapping results, mainly due to inaccurate camera poses. This paper presents OmniColor, a novel and efficient algorithm to colorize point clouds using an independent 360-degree camera. Given a LiDAR-based point cloud and a sequence of panorama images with initial coarse camera poses, our objective is to jointly optimize the poses of all frames for mapping images onto geometric reconstructions. Our pipeline works in an off-the-shelf manner that does not require any feature extraction or matching process. Instead, we find optimal poses by directly maximizing the photometric consistency of LiDAR maps. In experiments, we show that our method can overcome the severe visual distortion of omnidirectional images and greatly benefit from the wide field of view (FOV) of 360-degree cameras to reconstruct various scenarios with accuracy and stability. The code will be released at https://github.com/liubonan123/OmniColor/.

翻译：彩色点云作为一种简单高效的三维表示，在机器人导航和场景重建等领域具有诸多优势。当前，这种表示方式已广泛应用于依赖相机与LiDAR的三维重建任务。然而，现有许多框架在融合这两类传感器数据时效果欠佳，导致映射结果不理想，这主要归因于相机位姿不准确。本文提出OmniColor——一种利用独立360度相机对点云进行着色的新颖高效算法。给定基于LiDAR的点云及带有初始粗相机位姿的全景图像序列，我们的目标是在所有帧中联合优化位姿，以将图像映射到几何重建结果上。该流程可即插即用，无需任何特征提取或匹配过程。相反，我们通过直接最大化LiDAR地图的光度一致性来寻找最优位姿。实验表明，我们的方法能够克服全景图像的严重视觉畸变，并充分利用360度相机的宽视场角，以高精度和稳定性重建多种场景。代码将发布在https://github.com/liubonan123/OmniColor/。