Remarkable advances have been achieved recently in learning neural representations that characterize object geometry, while generating textured objects suitable for downstream applications and 3D rendering remains at an early stage. In particular, reconstructing textured geometry from images of real objects is a significant challenge -- reconstructed geometry is often inexact, making realistic texturing a significant challenge. We present Mesh2Tex, which learns a realistic object texture manifold from uncorrelated collections of 3D object geometry and photorealistic RGB images, by leveraging a hybrid mesh-neural-field texture representation. Our texture representation enables compact encoding of high-resolution textures as a neural field in the barycentric coordinate system of the mesh faces. The learned texture manifold enables effective navigation to generate an object texture for a given 3D object geometry that matches to an input RGB image, which maintains robustness even under challenging real-world scenarios where the mesh geometry approximates an inexact match to the underlying geometry in the RGB image. Mesh2Tex can effectively generate realistic object textures for an object mesh to match real images observations towards digitization of real environments, significantly improving over previous state of the art.

翻译：近年来，在表征物体几何形状的神经表示学习方面取得了显著进展，但生成适用于下游应用和三维渲染的带纹理物体仍处于早期阶段。特别是，从真实物体图像中重建带纹理的几何结构是一项重大挑战——重建的几何形状往往不精确，这使得逼真纹理生成成为难题。本文提出Mesh2Tex，通过利用混合网格-神经场纹理表示，从非关联的三维物体几何与逼真RGB图像集合中学习真实物体纹理流形。该纹理表示能够将高分辨率纹理紧凑编码为网格面重心坐标系下的神经场。学习到的纹理流形可有效导航，为给定的三维物体几何生成与输入RGB图像匹配的物体纹理，即使在网格几何形状与RGB图像中的底层几何形状近似不匹配的复杂真实场景中，仍能保持鲁棒性。Mesh2Tex能够为物体网格有效生成逼真纹理以匹配真实图像观测，助力真实环境数字化，显著超越现有最优方法。