Given a 3D object, kinematic motion prediction aims to identify the mobile parts as well as the corresponding motion parameters. Due to the large variations in both topological structure and geometric details of 3D objects, this remains a challenging task and the lack of large scale labeled data also constrain the performance of deep learning based approaches. In this paper, we tackle the task of object kinematic motion prediction problem in a semi-weakly supervised manner. Our key observations are two-fold. First, although 3D dataset with fully annotated motion labels is limited, there are existing datasets and methods for object part semantic segmentation at large scale. Second, semantic part segmentation and mobile part segmentation is not always consistent but it is possible to detect the mobile parts from the underlying 3D structure. Towards this end, we propose a graph neural network to learn the map between hierarchical part-level segmentation and mobile parts parameters, which are further refined based on geometric alignment. This network can be first trained on PartNet-Mobility dataset with fully labeled mobility information and then applied on PartNet dataset with fine-grained and hierarchical part-level segmentation. The network predictions yield a large scale of 3D objects with pseudo labeled mobility information and can further be used for weakly-supervised learning with pre-existing segmentation. Our experiments show there are significant performance boosts with the augmented data for previous method designed for kinematic motion prediction on 3D partial scans.

翻译：给定一个三维物体，运动学运动预测旨在识别可动部件及其对应的运动参数。由于三维物体在拓扑结构和几何细节上存在巨大差异，这一任务仍具有挑战性，而大规模标注数据的缺乏也限制了基于深度学习方法的表现。本文以半弱监督方式处理物体运动学运动预测问题。我们的关键发现有两方面：首先，虽然具有完整运动标注的三维数据集有限，但现有数据集和方法可大规模进行物体部件语义分割；其次，语义部件分割与可动部件分割并不总是一致，但可以从底层三维结构中检测可动部件。为此，我们提出一种图神经网络，学习层次化部件级分割与可动部件参数之间的映射，并基于几何对齐进一步优化。该网络可先在具有完整运动标注信息的PartNet-Mobility数据集上训练，然后应用于具有细粒度层次化部件级分割的PartNet数据集。网络预测生成大量带有伪运动标注的三维物体，并可用于基于预分割的弱监督学习。实验表明，针对三维局部扫描的运动学运动预测方法，通过增强数据获得了显著的性能提升。