We introduce 3DShape2VecSet, a novel shape representation for neural fields designed for generative diffusion models. Our shape representation can encode 3D shapes given as surface models or point clouds, and represents them as neural fields. The concept of neural fields has previously been combined with a global latent vector, a regular grid of latent vectors, or an irregular grid of latent vectors. Our new representation encodes neural fields on top of a set of vectors. We draw from multiple concepts, such as the radial basis function representation and the cross attention and self-attention function, to design a learnable representation that is especially suitable for processing with transformers. Our results show improved performance in 3D shape encoding and 3D shape generative modeling tasks. We demonstrate a wide variety of generative applications: unconditioned generation, category-conditioned generation, text-conditioned generation, point-cloud completion, and image-conditioned generation.

翻译：我们提出3DShape2VecSet，一种面向生成扩散模型的新型神经场形状表征方法。该表征能够将表面模型或点云形式的三维形状编码为神经场。此前，神经场概念已与全局潜在向量、规则潜在向量网格或不规则潜在向量网格相结合。本研究提出的新表征基于向量集对神经场进行编码。我们融合了径向基函数表征、交叉注意力与自注意力函数等多重概念，设计出特别适合Transformer处理的可学习表征方法。实验表明，该方法在三维形状编码与三维形状生成建模任务中均展现出更优性能。我们还展示了丰富的生成应用场景：无条件生成、类别条件生成、文本条件生成、点云补全及图像条件生成。