Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. Specifically, we extract keypoints and features from 2D image pairs using a frozen pretrained 2D backbone. These features are then projected in 3D, and patches are constructed by searching for neighboring points. We integrate the geometric and visual features of each point using our novel parameter-free geometric decoder. Subsequently, the task of determining correspondences between point clouds is formulated as an optimal transport problem. Extensive evaluations of ZeroReg demonstrate its competitive performance against both traditional and learning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.

翻译：基于学习的点云配准方法已显著超越传统方法，但其通常需要在特定数据集上进行大量训练。本文提出首个零样本点云配准方法ZeroReg，该方法无需在点云数据集上训练。ZeroReg的核心创新在于将图像关键点特征迁移至点云，并通过聚合三维几何邻域信息进行增强。具体而言，我们利用冻结的预训练二维骨干网络从二维图像对中提取关键点与特征，然后将这些特征投影至三维空间，通过搜索邻域点构建局部块。通过我们提出的无参数几何解码器，将每个点的几何特征与视觉特征进行融合。随后，点云间对应关系求解被建模为最优传输问题。大量评估表明，ZeroReg在传统方法与学习方法中均展现出竞争优势。在3DMatch、3DLoMatch和ScanNet等基准测试中，ZeroReg分别取得了超过84%、46%和75%的令人瞩目的召回率（RR）。