Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.

翻译：纹理合成是计算机图形学中的一个基本问题，其应用将惠及众多领域。现有方法在处理二维图像纹理方面表现有效。然而，许多真实世界的纹理（如草地、树叶和织物）在三维几何空间中包含细观结构，仅使用二维图像纹理无法有效建模。我们提出了一种基于神经辐射场（NeRF）的新型纹理合成方法，用于从给定的多视角图像中捕获并合成纹理。在所提出的NeRF纹理表示中，具有精细几何细节的场景被解耦为细观结构纹理和底层基础形状。这使得具有细观结构的纹理能够作为位于基础形状上的潜在特征被有效学习，这些特征被输入到同时训练的NeRF解码器中，以表示丰富的视角相关外观。利用这种隐式表示，我们可以通过潜在特征的块匹配来合成基于NeRF的纹理。然而，重建内容空间与潜在特征空间的度量标准之间的不一致可能会影响合成质量。为了提升匹配性能，我们通过引入聚类约束进一步规范化潜在特征的分布。除了在平面域上生成NeRF纹理外，我们的方法还能在曲面上合成NeRF纹理，这具有实际应用价值。实验结果与评估证明了我们方法的有效性。