While surface-based view synthesis algorithms are appealing due to their low computational requirements, they often struggle to reproduce thin structures. In contrast, more expensive methods that model the scene's geometry as a volumetric density field (e.g. NeRF) excel at reconstructing fine geometric detail. However, density fields often represent geometry in a "fuzzy" manner, which hinders exact localization of the surface. In this work, we modify density fields to encourage them to converge towards surfaces, without compromising their ability to reconstruct thin structures. First, we employ a discrete opacity grid representation instead of a continuous density field, which allows opacity values to discontinuously transition from zero to one at the surface. Second, we anti-alias by casting multiple rays per pixel, which allows occlusion boundaries and subpixel structures to be modelled without using semi-transparent voxels. Third, we minimize the binary entropy of the opacity values, which facilitates the extraction of surface geometry by encouraging opacity values to binarize towards the end of training. Lastly, we develop a fusion-based meshing strategy followed by mesh simplification and appearance model fitting. The compact meshes produced by our model can be rendered in real-time on mobile devices and achieve significantly higher view synthesis quality compared to existing mesh-based approaches.
翻译:基于表面的视图合成算法因其低计算需求而备受青睐,但在再现薄层结构时往往力不从心。相比之下,将场景几何建模为体积密度场(如NeRF)的代价更高的方法,在重建精细几何细节方面表现出色。然而,密度场常以“模糊”方式表征几何,这阻碍了表面的精确定位。本研究改进密度场,使其在保持薄层结构重建能力的同时向表面收敛。首先,我们采用离散不透明度网格替代连续密度场,使不透明度值能在表面处从零到一不连续跃迁。其次,通过每像素投射多射线进行抗锯齿处理,无需半透明体素即可建模遮挡边界与亚像素结构。第三,最小化不透明度值的二元熵,通过训练后期促使不透明度值二值化,从而简化表面几何提取。最后,我们开发了基于融合的网格化策略,辅以网格简化与外观模型拟合。本模型生成的紧凑网格可在移动设备上实时渲染,相比现有基于网格的方法,视图合成质量显著提升。