Realistic and diverse 3D shape generation is helpful for a wide variety of applications such as virtual reality, gaming, and animation. Modern generative models, such as GANs and diffusion models, learn from large-scale datasets and generate new samples following similar data distributions. However, when training data is limited, deep neural generative networks overfit and tend to replicate training samples. Prior works focus on few-shot image generation to produce high-quality and diverse results using a few target images. Unfortunately, abundant 3D shape data is typically hard to obtain as well. In this work, we make the first attempt to realize few-shot 3D shape generation by adapting generative models pre-trained on large source domains to target domains using limited data. To relieve overfitting and keep considerable diversity, we propose to maintain the probability distributions of the pairwise relative distances between adapted samples at feature-level and shape-level during domain adaptation. Our approach only needs the silhouettes of few-shot target samples as training data to learn target geometry distributions and achieve generated shapes with diverse topology and textures. Moreover, we introduce several metrics to evaluate the quality and diversity of few-shot 3D shape generation. The effectiveness of our approach is demonstrated qualitatively and quantitatively under a series of few-shot 3D shape adaptation setups.
翻译:逼真且多样化的3D形状生成对虚拟现实、游戏和动画等多种应用具有重要价值。现代生成模型(如生成对抗网络和扩散模型)通过大规模数据集学习,并生成符合相似数据分布的新样本。然而,当训练数据有限时,深度神经生成网络会出现过拟合,倾向于复制训练样本。先前的工作聚焦于小样本图像生成,利用少量目标图像生成高质量且多样化的结果。但遗憾的是,丰富的3D形状数据同样难以获取。在本工作中,我们首次尝试通过将在大规模源域上预训练的生成模型适配到目标域(仅使用有限数据),实现小样本3D形状生成。为缓解过拟合并保持足够多样性,我们提出在域适配过程中,维持适配样本在特征级和形状级上成对相对距离的概率分布。该方法仅需小样本目标样本的轮廓作为训练数据,即可学习目标几何分布,并生成具有多样拓扑结构和纹理的3D形状。此外,我们引入多项指标以评估小样本3D形状生成的质量和多样性。通过一系列小样本3D形状适配实验,我们的方法在定性和定量分析中均验证了有效性。