Modeling and synthesizing textures are essential for enhancing the realism of virtual environments. Methods that directly synthesize textures in 3D offer distinct advantages to the UV-mapping-based methods as they can create seamless textures and align more closely with the ways textures form in nature. We propose Mesh Neural Cellular Automata (MeshNCA), a method for directly synthesizing dynamic textures on 3D meshes without requiring any UV maps. MeshNCA is a generalized type of cellular automata that can operate on a set of cells arranged on a non-grid structure such as vertices of a 3D mesh. While only being trained on an Icosphere mesh, MeshNCA shows remarkable generalization and can synthesize textures on any mesh in real time after the training. Additionally, it accommodates multi-modal supervision and can be trained using different targets such as images, text prompts, and motion vector fields. Moreover, we conceptualize a way of grafting trained MeshNCA instances, enabling texture interpolation. Our MeshNCA model enables real-time 3D texture synthesis on meshes and allows several user interactions including texture density/orientation control, a grafting brush, and motion speed/direction control. Finally, we implement the forward pass of our MeshNCA model using the WebGL shading language and showcase our trained models in an online interactive demo which is accessible on personal computers and smartphones. Our demo and the high resolution version of this PDF are available at https://meshnca.github.io/.

翻译：建模和合成纹理对于增强虚拟环境的真实感至关重要。直接在三维空间中合成纹理的方法相较于基于UV映射的方法具有显著优势，因为它们能够生成无缝纹理，并更贴近自然界中纹理形成的方式。我们提出网格神经细胞自动机（MeshNCA），一种无需任何UV映射即可在三维网格上直接合成动态纹理的方法。MeshNCA是一种广义的细胞自动机，能够在非网格结构（如三维网格顶点）上排列的细胞集合中运行。尽管仅在二十面体网格上进行训练，MeshNCA展现出卓越的泛化能力，能够在训练后实时合成任意网格上的纹理。此外，它支持多模态监督，可利用图像、文本提示和运动向量场等不同目标进行训练。我们进一步构思了一种嫁接已训练MeshNCA实例的方法，从而实现纹理插值。我们的MeshNCA模型支持网格上的实时三维纹理合成，并允许多种用户交互，包括纹理密度/方向控制、嫁接画笔以及运动速度/方向控制。最后，我们使用WebGL着色语言实现了MeshNCA模型的前向传播，并在一个在线交互演示中展示了训练后的模型，该演示可在个人电脑和智能手机上访问。我们的演示及本PDF的高分辨率版本可见于https://meshnca.github.io/。