This work tackles the challenging task of achieving real-time novel view synthesis on various scenes, including highly reflective objects and unbounded outdoor scenes. Existing real-time rendering methods, especially those based on meshes, often have subpar performance in modeling surfaces with rich view-dependent appearances. Our key idea lies in leveraging meshes for rendering acceleration while incorporating a novel approach to parameterize view-dependent information. We decompose the color into diffuse and specular, and model the specular color in the reflected direction based on a neural environment map. Our experiments demonstrate that our method achieves comparable reconstruction quality for highly reflective surfaces compared to state-of-the-art offline methods, while also efficiently enabling real-time rendering on edge devices such as smartphones.

翻译：本工作致力于实现包含高反射物体与无界户外场景在内的各类场景的实时新视角合成这一挑战性任务。现有实时渲染方法（特别是基于网格的方法）在建模具有丰富视角相关外观的表面时，性能往往不尽如人意。我们的核心思路在于：在利用网格加速渲染的同时，引入一种参数化视角相关信息的新方法。我们将颜色分解为漫反射分量与镜面反射分量，并基于神经环境贴图在反射方向上对镜面反射颜色进行建模。实验表明，对于高反射表面，我们的方法在重建质量上可与最先进的离线方法相媲美，同时能高效地在智能手机等边缘设备上实现实时渲染。