Synthesizing novel 3D models that resemble the input example has long been pursued by graphics artists and machine learning researchers. In this paper, we present Sin3DM, a diffusion model that learns the internal patch distribution from a single 3D textured shape and generates high-quality variations with fine geometry and texture details. Training a diffusion model directly in 3D would induce large memory and computational cost. Therefore, we first compress the input into a lower-dimensional latent space and then train a diffusion model on it. Specifically, we encode the input 3D textured shape into triplane feature maps that represent the signed distance and texture fields of the input. The denoising network of our diffusion model has a limited receptive field to avoid overfitting, and uses triplane-aware 2D convolution blocks to improve the result quality. Aside from randomly generating new samples, our model also facilitates applications such as retargeting, outpainting and local editing. Through extensive qualitative and quantitative evaluation, we show that our method outperforms prior methods in generation quality of 3D shapes.

翻译：合成与输入示例相似的新颖三维模型一直是图形艺术家和机器学习研究者追求的目标。本文提出Sin3DM——一种扩散模型，该模型从单个三维纹理形状中学习内部块分布，并生成具有精细几何与纹理细节的高质量变体。直接在三维空间训练扩散模型会导致巨大的内存与计算开销，因此我们首先将输入压缩至低维潜空间，随后在此空间上训练扩散模型。具体而言，我们将输入的三维纹理形状编码为表示输入符号距离场与纹理场的三平面特征图。扩散模型的去噪网络采用有限的感受野以避免过拟合，并利用三平面感知的二维卷积模块提升结果质量。除随机生成新样本外，我们的模型还支持重定向、外推与局部编辑等应用。通过广泛的定性与定量评估，我们证明所提方法在三维形状生成质量上优于现有方法。