Rendering novel view images is highly desirable for many applications. Despite recent progress, it remains challenging to render high-fidelity and view-consistent novel views of large-scale scenes from in-the-wild images with inevitable artifacts (e.g., motion blur). To this end, we develop a hybrid neural rendering model that makes image-based representation and neural 3D representation join forces to render high-quality, view-consistent images. Besides, images captured in the wild inevitably contain artifacts, such as motion blur, which deteriorates the quality of rendered images. Accordingly, we propose strategies to simulate blur effects on the rendered images to mitigate the negative influence of blurriness images and reduce their importance during training based on precomputed quality-aware weights. Extensive experiments on real and synthetic data demonstrate our model surpasses state-of-the-art point-based methods for novel view synthesis. The code is available at https://daipengwa.github.io/Hybrid-Rendering-ProjectPage.
翻译:渲染新视角图像在诸多应用中具有重要意义。尽管近期取得进展,但从包含不可避免的伪影(如运动模糊)的野外图像中,为大规模场景渲染出高保真且视角一致的新视图仍具挑战性。为此,我们开发了一种混合神经渲染模型,使基于图像的表示与神经三维表示协同作用,以生成高质量、视角一致的图像。此外,野外拍摄的图像不可避免地包含运动模糊等伪影,这会降低渲染图像的质量。据此,我们提出策略在渲染图像上模拟模糊效果,以减轻模糊图像的负面影响,并基于预计算的质量感知权重降低其在训练过程中的重要性。在真实与合成数据上的大量实验表明,我们的模型在新视角合成任务上超越了最先进的基于点的方法。代码发布于 https://daipengwa.github.io/Hybrid-Rendering-ProjectPage。