We present the Locally Adaptive Morphable Model (LAMM), a highly flexible Auto-Encoder (AE) framework for learning to generate and manipulate 3D meshes. We train our architecture following a simple self-supervised training scheme in which input displacements over a set of sparse control vertices are used to overwrite the encoded geometry in order to transform one training sample into another. During inference, our model produces a dense output that adheres locally to the specified sparse geometry while maintaining the overall appearance of the encoded object. This approach results in state-of-the-art performance in both disentangling manipulated geometry and 3D mesh reconstruction. To the best of our knowledge LAMM is the first end-to-end framework that enables direct local control of 3D vertex geometry in a single forward pass. A very efficient computational graph allows our network to train with only a fraction of the memory required by previous methods and run faster during inference, generating 12k vertex meshes at $>$60fps on a single CPU thread. We further leverage local geometry control as a primitive for higher level editing operations and present a set of derivative capabilities such as swapping and sampling object parts. Code and pretrained models can be found at https://github.com/michaeltrs/LAMM.

翻译：我们提出了局部自适应可变形模型（LAMM），这是一个高度灵活的自编码器（AE）框架，用于学习生成和操控三维网格。我们遵循一个简单的自监督训练方案来训练我们的架构：通过一组稀疏控制顶点上的输入位移覆盖编码几何信息，从而将一个训练样本变换为另一个。在推理过程中，我们的模型生成密集输出，该输出在局部上严格遵循指定的稀疏几何结构，同时保持编码对象的整体外观。该方法在分离操控几何与三维网格重建方面均达到了最先进的性能。据我们所知，LAMM是首个能够在单次前向传递中实现三维顶点几何直接局部控制的端到端框架。一个非常高效的计算图允许我们的网络仅需先前方法所需内存的一小部分即可进行训练，并在推理时更快运行——在单CPU线程上以>60fps的速度生成12k顶点网格。我们进一步将局部几何控制作为更高级编辑操作的原语，并展示了一系列衍生能力，例如对象部件的交换与采样。代码和预训练模型可在https://github.com/michaeltrs/LAMM获取。