With the emergence of neural radiance fields (NeRFs), view synthesis quality has reached an unprecedented level. Compared to traditional mesh-based assets, this volumetric representation is more powerful in expressing scene geometry but inevitably suffers from high rendering costs and can hardly be involved in further processes like editing, posing significant difficulties in combination with the existing graphics pipeline. In this paper, we present a hybrid volume-mesh representation, VMesh, which depicts an object with a textured mesh along with an auxiliary sparse volume. VMesh retains the advantages of mesh-based assets, such as efficient rendering, compact storage, and easy editing, while also incorporating the ability to represent subtle geometric structures provided by the volumetric counterpart. VMesh can be obtained from multi-view images of an object and renders at 2K 60FPS on common consumer devices with high fidelity, unleashing new opportunities for real-time immersive applications.

翻译：随着神经辐射场（NeRF）的出现，视图合成质量达到了前所未有的高度。与传统基于网格的资产相比，这种体素表示在表达场景几何结构方面更强大，但不可避免地面临高渲染成本，且难以参与编辑等后续处理，与现有图形管线的结合存在显著困难。本文提出了一种混合体-网格表示——VMesh，它通过带纹理的网格与辅助稀疏体的组合来描述物体。VMesh既保留了网格基资产的优势，如高效渲染、紧凑存储和便捷编辑，又融入了体素表示对精细几何结构的表达能力。该表示可从物体的多视角图像中获取，并在普通消费设备上以2K 60FPS高保真度渲染，为实时沉浸式应用开辟了新机遇。