We address the problem of aligning real-world 3D data of garments, which benefits many applications such as texture learning, physical parameter estimation, generative modeling of garments, etc. Existing extrinsic methods typically perform non-rigid iterative closest point and struggle to align details due to incorrect closest matches and rigidity constraints. While intrinsic methods based on functional maps can produce high-quality correspondences, they work under isometric assumptions and become unreliable for garment deformations which are highly non-isometric. To achieve wrinkle-level as well as texture-level alignment, we present a novel coarse-to-fine two-stage method that leverages intrinsic manifold properties with two neural deformation fields, in the 3D space and the intrinsic space, respectively. The coarse stage performs a 3D fitting, where we leverage intrinsic manifold properties to define a manifold deformation field. The coarse fitting then induces a functional map that produces an alignment of intrinsic embeddings. We further refine the intrinsic alignment with a second neural deformation field for higher accuracy. We evaluate our method with our captured garment dataset, GarmCap. The method achieves accurate wrinkle-level and texture-level alignment and works for difficult garment types such as long coats. Our project page is https://jsnln.github.io/iccv2023_intrinsic/index.html.

翻译：我们研究了真实世界服装三维数据的对齐问题，该问题有助于纹理学习、物理参数估计、服装生成建模等多种应用。现有的外在方法通常执行非刚性迭代最近点算法，但由于错误的最近邻匹配和刚性约束，难以对齐细节。基于功能映射的内在方法虽然能生成高质量对应关系，但需在等距假设下工作，对于高度非等距的服装形变变得不可靠。为实现褶皱级和纹理级的对齐，我们提出了一种新颖的由粗到精的两阶段方法，该方法分别在三维空间和内在空间中利用两个神经形变场来发挥内在流形属性的作用。粗阶段执行三维拟合，利用内在流形属性定义了一个流形形变场。粗拟合随后生成一个功能映射，该映射产生内在嵌入的对齐。我们通过第二个神经形变场进一步优化内在对齐，以获得更高精度。我们使用自己采集的服装数据集GarmCap评估了该方法。该方法能够实现准确的褶皱级和纹理级对齐，并且适用于长外套等复杂服装类型。我们的项目页面为https://jsnln.github.io/iccv2023_intrinsic/index.html。