Current diffusion or flow-based generative models for 3D shapes divide to two: distilling pre-trained 2D image diffusion models, and training directly on 3D shapes. When training a diffusion or flow models on 3D shapes a crucial design choice is the shape representation. An effective shape representation needs to adhere three design principles: it should allow an efficient conversion of large 3D datasets to the representation form; it should provide a good tradeoff of approximation power versus number of parameters; and it should have a simple tensorial form that is compatible with existing powerful neural architectures. While standard 3D shape representations such as volumetric grids and point clouds do not adhere to all these principles simultaneously, we advocate in this paper a new representation that does. We introduce Mosaic-SDF (M-SDF): a simple 3D shape representation that approximates the Signed Distance Function (SDF) of a given shape by using a set of local grids spread near the shape's boundary. The M-SDF representation is fast to compute for each shape individually making it readily parallelizable; it is parameter efficient as it only covers the space around the shape's boundary; and it has a simple matrix form, compatible with Transformer-based architectures. We demonstrate the efficacy of the M-SDF representation by using it to train a 3D generative flow model including class-conditioned generation with the 3D Warehouse dataset, and text-to-3D generation using a dataset of about 600k caption-shape pairs.

翻译：当前基于扩散或流的3D形状生成模型分为两类：一类蒸馏预训练的2D图像扩散模型，另一类直接对3D形状进行训练。在3D形状上训练扩散或流模型时，一个关键设计选择是形状表征。有效的形状表征需遵循三个设计原则：应支持将大规模3D数据集高效转换为该表征形式；应在逼近能力与参数数量间提供良好权衡；应具备与现有强大神经架构兼容的简洁张量形式。尽管体素网格和点云等标准3D形状表征无法同时满足上述所有原则，本文提出了一种全新的表征方案。我们引入Mosaic-SDF（M-SDF）：一种简洁的3D形状表征，通过一组分布在形状边界附近的局部网格近似给定形状的符号距离函数。M-SDF表征可对每个形状独立快速计算，易于并行化；仅覆盖形状边界周围空间，参数效率高；且具有与Transformer架构兼容的简洁矩阵形式。我们通过训练3D生成流模型验证了M-SDF表征的有效性，包括基于3D Warehouse数据集的类别条件生成，以及利用约60万对文本-形状配对数据集实现的文本到3D生成。