Texture modeling and synthesis are essential for enhancing the realism of virtual environments. Methods that directly synthesize textures in 3D offer distinct advantages to the UV-mapping-based methods as they can create seamless textures and align more closely with the ways textures form in nature. We propose Mesh Neural Cellular Automata (MeshNCA), a method that directly synthesizes dynamic textures on 3D meshes without requiring any UV maps. MeshNCA is a generalized type of cellular automata that can operate on a set of cells arranged on non-grid structures such as the vertices of a 3D mesh. MeshNCA accommodates multi-modal supervision and can be trained using different targets such as images, text prompts, and motion vector fields. Only trained on an Icosphere mesh, MeshNCA shows remarkable test-time generalization and can synthesize textures on unseen meshes in real time. We conduct qualitative and quantitative comparisons to demonstrate that MeshNCA outperforms other 3D texture synthesis methods in terms of generalization and producing high-quality textures. Moreover, we introduce a way of grafting trained MeshNCA instances, enabling interpolation between textures. MeshNCA allows several user interactions including texture density/orientation controls, grafting/regenerate brushes, and motion speed/direction controls. Finally, we implement the forward pass of our MeshNCA model using the WebGL shading language and showcase our trained models in an online interactive demo, which is accessible on personal computers and smartphones and is available at https://meshnca.github.io.

翻译：纹理建模与合成对于提升虚拟环境的逼真度至关重要。相较于基于UV映射的方法，直接在3D空间中合成纹理具有显著优势，能够生成无缝纹理并更贴近自然界纹理形成方式。我们提出网格神经细胞自动机（MeshNCA），该方法无需任何UV映射即可直接在3D网格上合成动态纹理。MeshNCA是一种广义细胞自动机，可在非网格结构（如3D网格顶点）上排列的细胞集合中运行。该方法支持多模态监督，能够利用图像、文本提示和运动向量场等不同目标进行训练。即使仅使用二十面体网格进行训练，MeshNCA在测试阶段仍表现出卓越的泛化能力，可在未见过的网格上实时合成纹理。通过定性与定量比较，我们证明MeshNCA在泛化性能和高质量纹理生成方面均优于其他3D纹理合成方法。此外，我们提出一种嫁接训练后MeshNCA实例的方法，从而实现纹理间的插值过渡。MeshNCA支持多种用户交互操作，包括纹理密度/方向控制、嫁接/再生笔刷以及运动速度/方向控制。最后，我们使用WebGL着色语言实现MeshNCA模型的前向传播，并在在线交互演示中展示训练完成的模型——该演示可在个人计算机和智能手机上访问，网址为https://meshnca.github.io。