In this paper, we present TexPro, a novel method for high-fidelity material generation for input 3D meshes given text prompts. Unlike existing text-conditioned texture generation methods that typically generate RGB textures with baked lighting, TexPro is able to produce diverse texture maps via procedural material modeling, which enables physical-based rendering, relighting, and additional benefits inherent to procedural materials. Specifically, we first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model. We then derive texture maps through a rendering-based optimization with recent differentiable procedural materials. To this end, we design several techniques to handle the misalignment between the generated multi-view images and 3D meshes, and introduce a novel material agent that enhances material classification and matching by exploring both part-level understanding and object-aware material reasoning. Experiments demonstrate the superiority of the proposed method over existing SOTAs and its capability of relighting.

翻译：本文提出TexPro，一种基于文本提示为输入三维网格生成高保真材质的新方法。与现有通常生成带烘焙光照的RGB纹理的文本条件纹理生成方法不同，TexPro能够通过程序化材质建模生成多种纹理贴图，从而实现基于物理的渲染、重光照以及程序化材质固有的额外优势。具体而言，我们首先利用最新的文本到图像模型，根据输入文本提示生成多视角参考图像。随后，我们通过基于渲染的优化结合最新的可微分程序化材质来推导纹理贴图。为此，我们设计了多种技术来处理生成的多视角图像与三维网格之间的错位问题，并引入了一种新颖的材质代理，该代理通过探索部件级理解和对象感知的材质推理来增强材质分类与匹配。实验证明了所提方法相对于现有先进技术的优越性及其重光照能力。