In the context of image-to-point cloud registration, acquiring point-to-pixel correspondences presents a challenging task since the similarity between individual points and pixels is ambiguous due to the visual differences in data modalities. Nevertheless, the same object present in the two data formats can be readily identified from the local perspective of point sets and pixel patches. Motivated by this intuition, we propose a coarse-to-fine framework that emphasizes the establishment of correspondences between local point sets and pixel patches, followed by the refinement of results at both the point and pixel levels. On a coarse scale, we mimic the classic Visual Transformer to translate both image and point cloud into two sequences of local representations, namely point and pixel proxies, and employ attention to capture global and cross-modal contexts. To supervise the coarse matching, we propose a novel projected point proportion loss, which guides to match point sets with pixel patches where more points can be projected into. On a finer scale, point-to-pixel correspondences are then refined from a smaller search space (i.e., the coarsely matched sets and patches) via well-designed sampling, attentional learning and fine matching, where sampling masks are embedded in the last two steps to mitigate the negative effect of sampling. With the high-quality correspondences, the registration problem is then resolved by EPnP algorithm within RANSAC. Experimental results on large-scale outdoor benchmarks demonstrate our superiority over existing methods.

翻译：在图像到点云配准任务中，获取点与像素间的对应关系极具挑战性，这是由于不同数据模态间的视觉差异导致单个点与像素之间的相似性存在歧义。然而，两种数据格式中存在的同一物体可从点集与像素块的局部视角中轻松识别。受此直觉启发，我们提出一种由粗到细的框架，重点建立局部点集与像素块间的对应关系，随后在点级与像素级对结果进行精细化处理。在粗粒度层面，我们模拟经典视觉Transformer，将图像与点云分别转化为两组局部表征序列（即点代理与像素代理），并利用注意力机制捕捉全局及跨模态上下文。为监督粗匹配，我们提出一种新颖的投影点比例损失函数，引导点集与更多可投影点对应的像素块进行匹配。在细粒度层面，通过精心设计的采样、注意力学习与精细匹配，从缩小后的搜索空间（即粗匹配的集合与像素块）中进一步细化点-像素对应关系；其中，采样掩码嵌入后两个步骤以减少采样的负面影响。基于高质量对应关系，配准问题最终通过RANSAC框架内的EPnP算法求解。大规模室外基准数据集上的实验结果表明，本方法优于现有方法。