We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker's motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.

翻译：我们提出了一种新颖的基于局部性的学习方法，用于清理和求解光学动作捕捉数据。针对含噪标记点数据，我们提出了一种新型异构图神经网络，该网络将标记点和关节视为不同类型的节点，并利用图卷积操作提取标记点和关节的局部特征，进而将其转换为干净的动捕数据。处理异常标记点（例如被遮挡或存在较大追踪误差）的关键洞见在于：标记点的运动与其邻近标记点运动呈现强相关性，而与其他标记点相关性较弱，即所谓的局部性特性，这使我们能够高效地填补缺失标记点（例如由遮挡导致）。此外，我们还通过分析加速度曲线来识别由追踪误差导致的标记点异常值。最后，我们提出了一种基于表征学习和数据增强的训练范式，通过在带有掩码的数据上训练模型来实现。掩码方案旨在模拟真实数据中常见的遮挡和噪声标记点。实验表明，我们的方法在多种数据集的多项指标上均取得了高精度。广泛对比显示，本方法在遮挡标记点位置预测误差上相较现有最优方法提升了约20%，进而使重建的关节旋转和位置误差降低了30%。本文代码与数据可在 https://github.com/non-void/LocalMoCap 获取。