Estimating 3D full-body pose from sparse sensor data is a pivotal technique employed for the reconstruction of realistic human motions in Augmented Reality and Virtual Reality. However, translating sparse sensor signals into comprehensive human motion remains a challenge since the sparsely distributed sensors in common VR systems fail to capture the motion of full human body. In this paper, we use well-designed Body Pose Graph (BPG) to represent the human body and translate the challenge into a prediction problem of graph missing nodes. Then, we propose a novel full-body motion reconstruction framework based on BPG. To establish BPG, nodes are initially endowed with features extracted from sparse sensor signals. Features from identifiable joint nodes across diverse sensors are amalgamated and processed from both temporal and spatial perspectives. Temporal dynamics are captured using the Temporal Pyramid Structure, while spatial relations in joint movements inform the spatial attributes. The resultant features serve as the foundational elements of the BPG nodes. To further refine the BPG, node features are updated through a graph neural network that incorporates edge reflecting varying joint relations. Our method's effectiveness is evidenced by the attained state-of-the-art performance, particularly in lower body motion, outperforming other baseline methods. Additionally, an ablation study validates the efficacy of each module in our proposed framework.

翻译：从稀疏传感器数据估计3D全身姿态是增强现实与虚拟现实中重建逼真人体运动的关键技术。然而，由于常见VR系统中稀疏分布的传感器无法捕捉完整人体运动，将稀疏传感信号转化为全面人体运动仍是一项挑战。本文采用精心设计的人体姿态图（Body Pose Graph, BPG）表征人体，将这一挑战转化为图缺失节点预测问题，并据此提出一种基于BPG的新型全身运动重建框架。在构建BPG时，节点初始特征由稀疏传感信号提取而来。来自不同传感器的可识别关节节点特征经融合后，从时间与空间两个维度进行处理：采用时间金字塔结构捕捉时间动态特性，同时通过关节运动的空间关系获取空间属性，所得特征作为BPG节点的基本元素。为进一步优化BPG，节点特征通过融合反映不同关节关系的边信息的图神经网络进行更新。本方法的有效性通过在下半身运动等任务中取得的最优性能得到证实，其表现优于其他基线方法。此外，消融实验验证了框架中各模块的有效性。