We present a novel appearance model that simultaneously realizes explicit high-quality 3D surface mesh recovery and photorealistic novel view synthesis from sparse view samples. Our key idea is to model the underlying scene geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels (MAtCha Gaussians). MAtCha distills high-frequency scene surface details from an off-the-shelf monocular depth estimator and refines it through Gaussian surfel rendering. The Gaussian surfels are attached to the charts on the fly, satisfying photorealism of neural volumetric rendering and crisp geometry of a mesh model, i.e., two seemingly contradicting goals in a single model. At the core of MAtCha lies a novel neural deformation model and a structure loss that preserve the fine surface details distilled from learned monocular depths while addressing their fundamental scale ambiguities. Results of extensive experimental validation demonstrate MAtCha's state-of-the-art quality of surface reconstruction and photorealism on-par with top contenders but with dramatic reduction in the number of input views and computational time. We believe MAtCha will serve as a foundational tool for any visual application in vision, graphics, and robotics that require explicit geometry in addition to photorealism. Our project page is the following: https://anttwo.github.io/matcha/

翻译：我们提出了一种新颖的外观模型，能够同时从稀疏视图样本中实现显式高质量三维表面网格重建与照片级真实感的新视角合成。我们的核心思想是将底层场景几何网格建模为图册图谱，并通过二维高斯面元进行渲染（MAtCha Gaussians）。MAtCha从现成的单目深度估计器中提取高频场景表面细节，并通过高斯面元渲染进行优化。高斯面元在渲染过程中动态附着于图谱单元，在单一模型中同时满足神经体积渲染的照片级真实感与网格模型的清晰几何结构——这两个看似矛盾的目标。MAtCha的核心在于新颖的神经变形模型和结构损失函数，该设计在解决学习型单目深度固有尺度模糊性的同时，保留了从中所提取的精细表面细节。大量实验验证结果表明，MAtCha在表面重建质量方面达到最先进水平，其照片级真实感与顶尖模型相当，但所需输入视图数量和计算时间大幅减少。我们相信MAtCha将成为视觉、图形学和机器人学等领域中任何需要显式几何与照片级真实感并重的视觉应用的基础工具。项目页面如下：https://anttwo.github.io/matcha/