In the fields of computer graphics, computer vision and photogrammetry, Neural Radiance Fields (NeRFs) are a major topic driving current research and development. However, the quality of NeRF-generated 3D scene reconstructions and subsequent surface reconstructions, heavily relies on the network output, particularly the density. Regarding this critical aspect, we propose to utilize NeRF-Ensembles that provide a density uncertainty estimate alongside the mean density. We demonstrate that data constraints such as low-quality images and poses lead to a degradation of the training process, increased density uncertainty and decreased predicted density. Even with high-quality input data, the density uncertainty varies based on scene constraints such as acquisition constellations, occlusions and material properties. NeRF-Ensembles not only provide a tool for quantifying the uncertainty but exhibit two promising advantages: Enhanced robustness and artifact removal. Through the utilization of NeRF-Ensembles instead of single NeRFs, small outliers are removed, yielding a smoother output with improved completeness of structures. Furthermore, applying percentile-based thresholds on density uncertainty outliers proves to be effective for the removal of large (foggy) artifacts in post-processing. We conduct our methodology on 3 different datasets: (i) synthetic benchmark dataset, (ii) real benchmark dataset, (iii) real data under realistic recording conditions and sensors.
翻译:在计算机图形学、计算机视觉与摄影测量领域,神经辐射场(NeRF)是驱动当前研究与发展的重要课题。然而,NeRF生成的三维场景重建及其后续表面重建的质量高度依赖于网络输出,尤其是密度参数。针对这一关键问题,本文提出利用NeRF集成模型,在提供平均密度的同时给出密度不确定性估计。研究表明,低质量图像与位姿等数据约束会导致训练过程退化、密度不确定性增加及预测密度下降。即使使用高质量输入数据,密度不确定性仍会随采集构型、遮挡及材料属性等场景约束而变化。NeRF集成模型不仅提供了量化不确定性的工具,还展现出两个显著优势:增强鲁棒性与伪影消除。相较于单一NeRF模型,采用NeRF集成模型可消除微小异常值,获得更平滑的输出并提升结构完整性。此外,在后期处理中基于百分位数对密度不确定性异常值设定阈值,可有效消除大型(雾状)伪影。我们在三个不同数据集上验证该方法:(i) 合成基准数据集,(ii) 真实基准数据集,(iii) 在真实记录条件与传感器下获取的现实数据。