The main challenges of 3D pose transfer are: 1) Lack of paired training data with different characters performing the same pose; 2) Disentangling pose and shape information from the target mesh; 3) Difficulty in applying to meshes with different topologies. We thus propose a novel weakly-supervised keypoint-based framework to overcome these difficulties. Specifically, we use a topology-agnostic keypoint detector with inverse kinematics to compute transformations between the source and target meshes. Our method only requires supervision on the keypoints, can be applied to meshes with different topologies and is shape-invariant for the target which allows extraction of pose-only information from the target meshes without transferring shape information. We further design a cycle reconstruction to perform self-supervised pose transfer without the need for ground truth deformed mesh with the same pose and shape as the target and source, respectively. We evaluate our approach on benchmark human and animal datasets, where we achieve superior performance compared to the state-of-the-art unsupervised approaches and even comparable performance with the fully supervised approaches. We test on the more challenging Mixamo dataset to verify our approach's ability in handling meshes with different topologies and complex clothes. Cross-dataset evaluation further shows the strong generalization ability of our approach.

翻译：三维姿态迁移的主要挑战包括：1) 缺乏不同角色执行相同姿态的配对训练数据；2) 从目标网格中解耦姿态与形状信息；3) 难以应用于具有不同拓扑结构的网格。为此，我们提出了一种新颖的基于关键点的弱监督框架来克服这些困难。具体而言，我们采用拓扑无关的关键点检测器结合逆运动学计算源网格与目标网格之间的变换。该方法仅需对关键点进行监督，可应用于不同拓扑结构的网格，且对目标具有形状不变性，从而能从目标网格中提取纯姿态信息而无需迁移形状信息。我们进一步设计了循环重建机制，实现无需真实形变网格（分别与目标和源具有相同姿态和形状）的自监督姿态迁移。在基准人体和动物数据集上的评估表明，该方法性能优于现有最先进的无监督方法，甚至可与全监督方法相媲美。我们在更具挑战性的Mixamo数据集上进行了测试，验证了该方法处理不同拓扑结构和复杂服装网格的能力。跨数据集评估进一步证明了我们方法强大的泛化能力。