Neural Radiance Fields (NeRF) have achieved photorealistic novel views synthesis; however, the requirement of accurate camera poses limits its application. Despite analysis-by-synthesis extensions for jointly learning neural 3D representations and registering camera frames exist, they are susceptible to suboptimal solutions if poorly initialized. We propose L2G-NeRF, a Local-to-Global registration method for bundle-adjusting Neural Radiance Fields: first, a pixel-wise flexible alignment, followed by a frame-wise constrained parametric alignment. Pixel-wise local alignment is learned in an unsupervised way via a deep network which optimizes photometric reconstruction errors. Frame-wise global alignment is performed using differentiable parameter estimation solvers on the pixel-wise correspondences to find a global transformation. Experiments on synthetic and real-world data show that our method outperforms the current state-of-the-art in terms of high-fidelity reconstruction and resolving large camera pose misalignment. Our module is an easy-to-use plugin that can be applied to NeRF variants and other neural field applications. The Code and supplementary materials are available at https://rover-xingyu.github.io/L2G-NeRF/.

翻译：神经辐射场（NeRF）实现了照片级真实感的新视角合成，但其对精确相机位姿的要求限制了应用。尽管存在通过分析-合成扩展来联合学习神经三维表示与配准相机帧的方法，若初始值不佳，这些方法易陷入次优解。我们提出L2G-NeRF——一种用于捆绑调整神经辐射场的局部到全局配准方法：首先进行逐像素柔性对齐，随后进行逐帧约束参数化对齐。逐像素局部对齐通过深度网络以无监督方式学习，优化光度重建误差；逐帧全局对齐则利用可微参数估计求解器对逐像素对应关系进行求解，以寻找全局变换。在合成与真实数据上的实验表明，本方法在高保真重建与解决大幅相机位姿偏差方面优于现有最先进技术。本模块作为易于使用的插件，可应用于NeRF变体及其他神经场应用。代码与补充材料见https://rover-xingyu.github.io/L2G-NeRF/。