We introduce Shape Tokens, a 3D representation that is continuous, compact, and easy to incorporate into machine learning models. Shape Tokens act as conditioning vectors that represent shape information in a 3D flow-matching model. The flow-matching model is trained to approximate probability density functions corresponding to delta functions concentrated on the surfaces of shapes in 3D. By attaching Shape Tokens to various machine learning models, we can generate new shapes, convert images to 3D, align 3D shapes with text and images, and render shapes directly at variable, user specified, resolution. Moreover, Shape Tokens enable a systematic analysis of geometric properties such as normal, density, and deformation field. Across all tasks and experiments, utilizing Shape Tokens demonstrate strong performance compared to existing baselines.
翻译:我们提出形状标记(Shape Tokens),这是一种连续、紧凑且易于集成到机器学习模型中的三维表示方法。形状标记作为条件向量,在三维流匹配模型中表征形状信息。该流匹配模型经过训练,能够近似对应于三维空间中形状表面上的狄拉克函数概率密度分布。通过将形状标记与多种机器学习模型结合,我们能够生成新形状、将图像转换为三维模型、实现三维形状与文本及图像的对齐,并以用户指定的可变分辨率直接渲染形状。此外,形状标记支持对法向量、密度场及变形场等几何特性进行系统性分析。在所有任务与实验中,使用形状标记相较于现有基线方法均展现出卓越性能。