The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools. Here, we focus on inpainting regions in a view-consistent and controllable manner. In addition to the typical NeRF inputs and masks delineating the unwanted region in each view, we require only a single inpainted view of the scene, i.e., a reference view. We use monocular depth estimators to back-project the inpainted view to the correct 3D positions. Then, via a novel rendering technique, a bilateral solver can construct view-dependent effects in non-reference views, making the inpainted region appear consistent from any view. For non-reference disoccluded regions, which cannot be supervised by the single reference view, we devise a method based on image inpainters to guide both the geometry and appearance. Our approach shows superior performance to NeRF inpainting baselines, with the additional advantage that a user can control the generated scene via a single inpainted image. Project page: https://ashmrz.github.io/reference-guided-3d

翻译：神经辐射场（NeRF）在视图合成中的普及催生了对其编辑工具的需求。本文聚焦于以视图一致且可控的方式修复区域。除典型的NeRF输入和每视图标注待修复区域的掩码外，我们仅需场景的单个修复视图（即参考视图）。我们利用单目深度估计器将修复视图反投影至正确的三维位置。随后，通过一种新颖的渲染技术，双边求解器可在非参考视图中构建与视角相关的效果，使修复区域从任意视角观察均保持一致性。针对无法由单一参考视图监督的非参考视图遮挡区域，我们设计了一种基于图像修复器的方法来同时指导几何与外观。我们的方法在性能上优于NeRF修复基线方法，并具备额外优势：用户可通过单张修复图像控制生成场景。项目页面：https://ashmrz.github.io/reference-guided-3d