We present a novel approach for synthesizing realistic novel views using Neural Radiance Fields (NeRF) with uncontrolled photos in the wild. While NeRF has shown impressive results in controlled settings, it struggles with transient objects commonly found in dynamic and time-varying scenes. Our framework called \textit{Inpainting Enhanced NeRF}, or \ours, enhances the conventional NeRF by drawing inspiration from the technique of image inpainting. Specifically, our approach extends the Multi-Layer Perceptrons (MLP) of NeRF, enabling it to simultaneously generate intrinsic properties (static color, density) and extrinsic transient masks. We introduce an inpainting module that leverages the transient masks to effectively exclude occlusions, resulting in improved volume rendering quality. Additionally, we propose a new training strategy with frequency regularization to address the sparsity issue of low-frequency transient components. We evaluate our approach on internet photo collections of landmarks, demonstrating its ability to generate high-quality novel views and achieve state-of-the-art performance.

翻译：本文提出了一种利用野外无约束照片合成逼真新视角的新方法，该方法基于神经辐射场（NeRF）。尽管NeRF在受控环境中已展现出令人印象深刻的效果，但在动态时变场景中普遍存在的瞬态物体处理方面仍存在困难。我们提出的框架称为“修复增强神经辐射场”（简称IE-NeRF），通过借鉴图像修复技术的思想对传统NeRF进行了增强。具体而言，我们的方法扩展了NeRF的多层感知机（MLP）结构，使其能够同时生成内在属性（静态颜色、密度）和外在瞬态掩码。我们引入了一个修复模块，该模块利用瞬态掩码有效排除遮挡物，从而提升体渲染质量。此外，针对低频瞬态成分的稀疏性问题，我们提出了一种结合频率正则化的新型训练策略。通过在互联网地标照片集上进行评估，我们证明了该方法能够生成高质量的新视角图像，并达到最先进的性能水平。