We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker's motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.

翻译：我们提出了一种基于局域性的新型学习方法，用于清洁与求解光学运动捕捉数据。针对带噪声的标记点数据，我们设计了一种新的异构图神经网络，将标记点与关节视为不同类型的节点，并通过图卷积操作提取标记点与关节的局部特征，进而将其转化为清洁的运动数据。为处理异常标记点（例如被遮挡或存在较大追踪误差的情况），核心思路在于：标记点的运动与其相邻标记点的运动呈现强相关性，而与其他标记点的相关性较弱——即局域性——这使得我们能够高效地填充缺失标记点（例如因遮挡导致）。此外，我们还通过分析标记点的加速度曲线来识别由追踪误差导致的异常值。最终，我们提出了一种基于表征学习与数据增强的训练机制，通过在带掩码的数据上训练模型实现。掩码方案旨在模拟真实数据中常见的遮挡与噪声标记点。实验结果表明，本方法在多个数据集的多项指标上均达到了高精度。广泛对比显示，本方法在遮挡标记点位置误差的预测精度上比现有最先进方法提升约20%，进而使重构的关节旋转与位置误差降低30%。本文的代码与数据可在 https://github.com/non-void/LocalMoCap 获取。