Neural Radiance Fields (NeRF) have shown impressive novel view synthesis results; nonetheless, even thorough recordings yield imperfections in reconstructions, for instance due to poorly observed areas or minor lighting changes. Our goal is to mitigate these imperfections from various sources with a joint solution: we take advantage of the ability of generative adversarial networks (GANs) to produce realistic images and use them to enhance realism in 3D scene reconstruction with NeRFs. To this end, we learn the patch distribution of a scene using an adversarial discriminator, which provides feedback to the radiance field reconstruction, thus improving realism in a 3D-consistent fashion. Thereby, rendering artifacts are repaired directly in the underlying 3D representation by imposing multi-view path rendering constraints. In addition, we condition a generator with multi-resolution NeRF renderings which is adversarially trained to further improve rendering quality. We demonstrate that our approach significantly improves rendering quality, e.g., nearly halving LPIPS scores compared to Nerfacto while at the same time improving PSNR by 1.4dB on the advanced indoor scenes of Tanks and Temples.

翻译：神经辐射场（NeRF）在新视角合成方面取得了显著成果；然而，即便是最详尽的记录，由于观察不充分的区域或细微光照变化等原因，重建结果仍存在缺陷。我们的目标是通过一个联合解决方案来减轻这些来自不同来源的缺陷：利用生成对抗网络（GAN）生成逼真图像的能力，将其用于增强NeRF三维场景重建的真实感。为此，我们使用对抗判别器学习场景的块分布，该判别器为辐射场重建提供反馈，从而以三维一致的方式提升真实感。通过施加多视角路径渲染约束，渲染伪影直接在底层三维表示中得到修复。此外，我们用多分辨率NeRF渲染结果对生成器进行条件约束，并通过对抗训练进一步提升渲染质量。我们证明，该方法显著提高了渲染质量，例如，在Tanks and Temples的复杂室内场景中，与Nerfacto相比，LPIPS分数几乎减半，同时PSNR提升了1.4dB。