This paper focuses on training a robust RGB-D registration model without ground-truth pose supervision. Existing methods usually adopt a pairwise training strategy based on differentiable rendering, which enforces the photometric and the geometric consistency between the two registered frames as supervision. However, this frame-to-frame framework suffers from poor multi-view consistency due to factors such as lighting changes, geometry occlusion and reflective materials. In this paper, we present NeRF-UR, a novel frame-to-model optimization framework for unsupervised RGB-D registration. Instead of frame-to-frame consistency, we leverage the neural radiance field (NeRF) as a global model of the scene and use the consistency between the input and the NeRF-rerendered frames for pose optimization. This design can significantly improve the robustness in scenarios with poor multi-view consistency and provides better learning signal for the registration model. Furthermore, to bootstrap the NeRF optimization, we create a synthetic dataset, Sim-RGBD, through a photo-realistic simulator to warm up the registration model. By first training the registration model on Sim-RGBD and later unsupervisedly fine-tuning on real data, our framework enables distilling the capability of feature extraction and registration from simulation to reality. Our method outperforms the state-of-the-art counterparts on two popular indoor RGB-D datasets, ScanNet and 3DMatch. Code and models will be released for paper reproduction.

翻译：本文聚焦于在没有真实位姿监督的情况下训练鲁棒的RGB-D配准模型。现有方法通常采用基于可微分渲染的成对训练策略，通过强制两配准帧之间的光度一致性和几何一致性作为监督信号。然而，这种帧到帧框架因光照变化、几何遮挡和反射材料等因素导致多视角一致性较差。本文提出NeRF-UR，一种用于无监督RGB-D配准的新型帧到模型优化框架。不同于帧到帧一致性，我们利用神经辐射场（NeRF）作为场景的全局模型，通过输入帧与NeRF重渲染帧之间的一致性进行位姿优化。该设计能显著提升在多视角一致性较差场景下的鲁棒性，并为配准模型提供更优的学习信号。此外，为启动NeRF优化，我们通过逼真模拟器创建合成数据集Sim-RGBD，用于预热配准模型。通过先在Sim-RGBD上训练配准模型，再在真实数据上进行无监督微调，我们的框架实现了从仿真到现实的的特征提取与配准能力蒸馏。在两个主流室内RGB-D数据集（ScanNet和3DMatch）上，我们的方法优于现有最先进方案。代码与模型将公开发布以复现论文结果。