We introduce 3DShape2VecSet, a novel shape representation for neural fields designed for generative diffusion models. Our shape representation can encode 3D shapes given as surface models or point clouds, and represents them as neural fields. The concept of neural fields has previously been combined with a global latent vector, a regular grid of latent vectors, or an irregular grid of latent vectors. Our new representation encodes neural fields on top of a set of vectors. We draw from multiple concepts, such as the radial basis function representation and the cross attention and self-attention function, to design a learnable representation that is especially suitable for processing with transformers. Our results show improved performance in 3D shape encoding and 3D shape generative modeling tasks. We demonstrate a wide variety of generative applications: unconditioned generation, category-conditioned generation, text-conditioned generation, point-cloud completion, and image-conditioned generation.

翻译：我们提出3DShape2VecSet，一种专为生成扩散模型设计的新型神经场形状表征。该表征能够对以表面模型或点云形式呈现的三维形状进行编码，并将其表示为神经场。此前，神经场概念已与全局潜在向量、规则网格潜在向量或不规则网格潜在向量相结合。我们的新表征基于向量集合构建神经场，融合径向基函数表征、交叉注意力机制与自注意力机制等多项核心技术，设计出特别适用于Transformer架构处理的可学习表征。实验表明，该方法在三维形状编码与生成建模任务中均取得更优性能。我们展示了丰富的生成应用场景：无条件生成、类别条件生成、文本条件生成、点云补全以及图像条件生成。