Given a 3D object, kinematic motion prediction aims to identify the mobile parts as well as the corresponding motion parameters. Due to the large variations in both topological structure and geometric details of 3D objects, this remains a challenging task and the lack of large scale labeled data also constrain the performance of deep learning based approaches. In this paper, we tackle the task of object kinematic motion prediction problem in a semi-weakly supervised manner. Our key observations are two-fold. First, although 3D dataset with fully annotated motion labels is limited, there are existing datasets and methods for object part semantic segmentation at large scale. Second, semantic part segmentation and mobile part segmentation is not always consistent but it is possible to detect the mobile parts from the underlying 3D structure. Towards this end, we propose a graph neural network to learn the map between hierarchical part-level segmentation and mobile parts parameters, which are further refined based on geometric alignment. This network can be first trained on PartNet-Mobility dataset with fully labeled mobility information and then applied on PartNet dataset with fine-grained and hierarchical part-level segmentation. The network predictions yield a large scale of 3D objects with pseudo labeled mobility information and can further be used for weakly-supervised learning with pre-existing segmentation. Our experiments show there are significant performance boosts with the augmented data for previous method designed for kinematic motion prediction on 3D partial scans.

翻译：给定一个三维物体，运动学运动预测旨在识别可动部件及其对应的运动参数。由于三维物体在拓扑结构和几何细节上存在巨大差异，这仍然是一项具有挑战性的任务，而大规模标注数据的缺乏也限制了基于深度学习方法的表现。本文以半弱监督方式处理物体运动学运动预测问题。我们的关键观察有两方面：首先，尽管完全标注运动标签的三维数据集有限，但现有的大规模物体部件语义分割数据集和方法已经存在；其次，语义部件分割与可动部件分割并不总是一致，但从底层三维结构中检测可动部件是可能的。为此，我们提出一种图神经网络来学习层次化部件级分割与可动部件参数之间的映射关系，并基于几何对齐进一步优化这些参数。该网络可首先在具有完整运动标注信息的PartNet-Mobility数据集上训练，随后应用于具有细粒度层次化部件级分割的PartNet数据集。网络预测产生大量带有伪标注运动信息的三维物体，并可进一步用于基于预分割的弱监督学习。实验表明，通过增强数据，针对三维局部扫描运动学运动预测的先前方法在性能上取得了显著提升。