In this paper, we introduce PCR-CG: a novel 3D point cloud registration module explicitly embedding the color signals into the geometry representation. Different from previous methods that only use geometry representation, our module is specifically designed to effectively correlate color into geometry for the point cloud registration task. Our key contribution is a 2D-3D cross-modality learning algorithm that embeds the deep features learned from color signals to the geometry representation. With our designed 2D-3D projection module, the pixel features in a square region centered at correspondences perceived from images are effectively correlated with point clouds. In this way, the overlapped regions can be inferred not only from point cloud but also from the texture appearances. Adding color is non-trivial. We compare against a variety of baselines designed for adding color to 3D, such as exhaustively adding per-pixel features or RGB values in an implicit manner. We leverage Predator [25] as the baseline method and incorporate our proposed module onto it. To validate the effectiveness of 2D features, we ablate different 2D pre-trained networks and show a positive correlation between the pre-trained weights and the task performance. Our experimental results indicate a significant improvement of 6.5% registration recall over the baseline method on the 3DLoMatch benchmark. We additionally evaluate our approach on SOTA methods and observe consistent improvements, such as an improvement of 2.4% registration recall over GeoTransformer as well as 3.5% over CoFiNet. Our study reveals a significant advantages of correlating explicit deep color features to the point cloud in the registration task.

翻译：本文提出PCR-CG：一种将颜色信号显式嵌入几何表示的新型3D点云配准模块。与仅使用几何表示的现有方法不同，本模块专门设计用于在点云配准任务中有效关联颜色与几何信息。核心贡献在于提出一种2D-3D跨模态学习算法，将颜色信号中学习到的深度特征嵌入几何表示。通过设计的2D-3D投影模块，图像中对应点周围方形区域的像素特征可与点云高效关联。由此，重叠区域不仅可通过点云推断，还可借助纹理外观进行判别。颜色信息的加入并非简单操作：我们比较了多种旨在为3D添加颜色的基线方法（例如以隐式方式逐像素添加特征或RGB值）。基于Predator[25]基线方法，我们将所提模块集成其中。为验证2D特征的有效性，我们消融不同2D预训练网络，并发现预训练权重与任务性能呈正相关。实验结果表明，在3DLoMatch基准测试上，本方法的配准召回率较基线方法显著提升6.5%。进一步在SOTA方法上的评估显示持续改进：相较GeoTransformer提升2.4%，较CoFiNet提升3.5%。本研究揭示了在配准任务中将显式深度颜色特征与点云关联的显著优势。