Reconstructing real-world 3D objects has numerous applications in computer vision, such as virtual reality, video games, and animations. Ideally, 3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time. Traditional methods match pixels between images using photo-consistency constraints or learned features, while differentiable rendering methods like Neural Radiance Fields (NeRF) use surface-based representations or differentiable volume rendering to generate high-fidelity scenes. However, these methods require excessive runtime for rendering, making them impractical for daily applications. To address these challenges, we present $\textbf{EvaSurf}$, an $\textbf{E}$fficient $\textbf{V}$iew-$\textbf{A}$ware Implicit Textured $\textbf{Surf}$ace Reconstruction method on Mobile Devices. In our method, we first employ an efficient surface-based model with a multi-view supervision module to ensure accurate mesh creation. To enable high-fidelity rendering, we learn an implicit texture embedded with a set of Gaussian lobes to capture view-dependent information. Furthermore, With the explicit geometry and the implicit texture, we can employ a lightweight neural shader to reduce the expense of computation and further support real-time rendering on common mobile devices. Extensive experiments demonstrate that our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets. Moreover, our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40FPS (Frames Per Second), with a final package required for rendering taking up only 40-50 MB.

翻译：重建真实世界3D物体在计算机视觉中具有众多应用，例如虚拟现实、视频游戏和动画。理想情况下，3D重建方法应实时生成具有3D一致性的高保真结果。传统方法使用光度一致性约束或学习特征进行像素匹配，而神经辐射场（NeRF）等可微渲染方法则使用基于表面的表示或可微体积渲染生成高保真场景。然而，这些方法需要过长的渲染时间，使其难以应用于日常场景。为解决这些挑战，我们提出$\textbf{EvaSurf}$——一种在移动设备上实现的$\textbf{E}$fficient $\textbf{V}$iew-$\textbf{A}$ware（高效视图感知）隐式纹理$\textbf{Surf}$ace（表面）重建方法。在我们的方法中，首先采用基于高效表面的模型，并结合多视图监督模块以确保精确的网格生成。为实现高保真渲染，我们学习嵌入一组高斯瓣的隐式纹理以捕捉视图依赖信息。此外，借助显式几何和隐式纹理，我们可以使用轻量级神经着色器降低计算开销，并进一步支持在普通移动设备上实时渲染。大量实验表明，我们的方法能够在合成数据集和真实数据集上重建高质量外观和精确网格。同时，我们的方法仅需使用单个GPU训练1-2小时，即可在移动设备上以超过40FPS（帧每秒）运行，最终渲染所需的包大小仅占用40-50 MB。