3D shapes have complementary abstractions from low-level geometry to part-based hierarchies to languages, which convey different levels of information. This paper presents a unified framework to translate between pairs of shape abstractions: $\textit{Text}$ $\Longleftrightarrow$ $\textit{Point Cloud}$ $\Longleftrightarrow$ $\textit{Program}$. We propose $\textbf{Neural Shape Compiler}$ to model the abstraction transformation as a conditional generation process. It converts 3D shapes of three abstract types into unified discrete shape code, transforms each shape code into code of other abstract types through the proposed $\textit{ShapeCode Transformer}$, and decodes them to output the target shape abstraction. Point Cloud code is obtained in a class-agnostic way by the proposed $\textit{Point}$VQVAE. On Text2Shape, ShapeGlot, ABO, Genre, and Program Synthetic datasets, Neural Shape Compiler shows strengths in $\textit{Text}$ $\Longrightarrow$ $\textit{Point Cloud}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Text}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Program}$, and Point Cloud Completion tasks. Additionally, Neural Shape Compiler benefits from jointly training on all heterogeneous data and tasks.

翻译：三维形状具有从低级几何到部件层级再到语言等多层次的互补抽象，传递不同粒度的信息。本文提出一种统一框架，用于实现形状抽象对之间的双向转换: $\textit{文本}$ $\Longleftrightarrow$ $\textit{点云}$ $\Longleftrightarrow$ $\textit{程序}$。我们提出$\textbf{神经形状编译器}$，将抽象转换建模为条件生成过程。该框架将三种抽象类型的三维形状转化为统一的离散形状编码，通过所提出的$\textit{ShapeCode Transformer}$将各形状编码转换为其他抽象类型的编码，并通过解码输出目标形状抽象。点云编码通过所提出的$\textit{Point}$VQVAE以类别无关方式获取。在Text2Shape、ShapeGlot、ABO、Genre和Program Synthetic数据集上，神经形状编译器在$\textit{文本}$ $\Longrightarrow$ $\textit{点云}$、$\textit{点云}$ $\Longrightarrow$ $\textit{文本}$、$\textit{点云}$ $\Longrightarrow$ $\textit{程序}$以及点云补全任务中展现出优势。此外，神经形状编译器得益于在所有异构数据和任务上的联合训练。