In this paper, we introduce PCR-CG: a novel 3D point cloud registration module explicitly embedding the color signals into the geometry representation. Different from previous methods that only use geometry representation, our module is specifically designed to effectively correlate color into geometry for the point cloud registration task. Our key contribution is a 2D-3D cross-modality learning algorithm that embeds the deep features learned from color signals to the geometry representation. With our designed 2D-3D projection module, the pixel features in a square region centered at correspondences perceived from images are effectively correlated with point clouds. In this way, the overlapped regions can be inferred not only from point cloud but also from the texture appearances. Adding color is non-trivial. We compare against a variety of baselines designed for adding color to 3D, such as exhaustively adding per-pixel features or RGB values in an implicit manner. We leverage Predator [25] as the baseline method and incorporate our proposed module onto it. To validate the effectiveness of 2D features, we ablate different 2D pre-trained networks and show a positive correlation between the pre-trained weights and the task performance. Our experimental results indicate a significant improvement of 6.5% registration recall over the baseline method on the 3DLoMatch benchmark. We additionally evaluate our approach on SOTA methods and observe consistent improvements, such as an improvement of 2.4% registration recall over GeoTransformer as well as 3.5% over CoFiNet. Our study reveals a significant advantages of correlating explicit deep color features to the point cloud in the registration task.

翻译：本文提出PCR-CG：一种新型三维点云配准模块，该模块将颜色信号显式嵌入几何表示中。与仅使用几何表示的既往方法不同，本模块专为点云配准任务设计，旨在有效建立颜色与几何之间的关联。我们的核心贡献在于提出了一种2D-3D跨模态学习算法，可将从颜色信号中学习到的深度特征嵌入几何表示。通过设计的2D-3D投影模块，图像中对应点周边方形区域的像素特征得以与点云有效关联。由此，重叠区域不仅可通过点云推断，还可依据纹理外观进行识别。引入颜色信息并非易事。我们对比了多种为三维数据添加颜色设计的基线方法（如以隐式方式穷尽添加逐像素特征或RGB值）。以Predator [25]为基准方法，我们将所提模块嵌入其中。为验证2D特征的有效性，我们对不同2D预训练网络进行消融实验，结果表明预训练权重与任务性能呈正相关。实验结果显示，在3DLoMatch基准上，本方法相较于基准方法的配准召回率提升6.5%。我们还在现有最优方法上评估本方法，观察到一致性能提升：相较于GeoTransformer提升2.4%配准召回率，相较于CoFiNet提升3.5%。研究表明，在配准任务中将显式深度颜色特征与点云关联具有显著优势。