Novel photo-realistic texture synthesis is an important task for generating novel scenes, including asset generation for 3D simulations. However, to date, these methods predominantly generate textured objects in 2D space. If we rely on 2D object generation, then we need to make a computationally expensive forward pass each time we change the camera viewpoint or lighting. Recent work that can generate textures in 3D requires 3D component segmentation that is expensive to acquire. In this work, we present a novel conditional generative architecture that we call a graph generative adversarial network (GGAN) that can generate textures in 3D by learning object component information in an unsupervised way. In this framework, we do not need an expensive forward pass whenever the camera viewpoint or lighting changes, and we do not need expensive 3D part information for training, yet the model can generalize to unseen 3D meshes and generate appropriate novel 3D textures. We compare this approach against state-of-the-art texture generation methods and demonstrate that the GGAN obtains significantly better texture generation quality (according to Frechet inception distance). We release our model source code as open source.

翻译：新颖的逼真纹理合成是生成新颖场景（包括三维仿真中的资产生成）的重要任务。然而，迄今为止，这些方法主要是在二维空间中生成带纹理的物体。若依赖二维物体生成，则每次改变摄像机视角或光照时，都需要进行一次计算成本高昂的前向传播。近期能够在三维空间中生成纹理的工作，则需要获取成本高昂的三维部件分割信息。在本工作中，我们提出一种新颖的条件生成架构——图生成对抗网络（GGAN），该网络能够以无监督方式学习物体部件信息，从而在三维空间中生成纹理。在此框架下，我们无需在摄像机视角或光照变化时进行昂贵的前向传播，也无需为训练获取昂贵的三维部件信息，但模型仍能泛化至未见过的三维网格，并生成合适的新颖三维纹理。我们将此方法与现有最先进的纹理生成方法进行对比，结果表明GGAN在纹理生成质量上显著更优（依据弗雷歇初始距离）。我们将模型源代码作为开源代码发布。