Learning-based approaches to monocular motion capture have recently shown promising results by learning to regress in a data-driven manner. However, due to the challenges in data collection and network designs, it remains challenging for existing solutions to achieve real-time full-body capture while being accurate in world space. In this work, we contribute a sequential proxy-to-motion learning scheme together with a proxy dataset of 2D skeleton sequences and 3D rotational motions in world space. Such proxy data enables us to build a learning-based network with accurate full-body supervision while also mitigating the generalization issues. For more accurate and physically plausible predictions, a contact-aware neural motion descent module is proposed in our network so that it can be aware of foot-ground contact and motion misalignment with the proxy observations. Additionally, we share the body-hand context information in our network for more compatible wrist poses recovery with the full-body model. With the proposed learning-based solution, we demonstrate the first real-time monocular full-body capture system with plausible foot-ground contact in world space. More video results can be found at our project page: https://liuyebin.com/proxycap.

翻译：基于学习的方法通过数据驱动回归在单目运动捕捉领域近期展现出令人瞩目的成果。然而，受限于数据采集难度和网络设计挑战，现有方案仍难以在实现世界空间高精度的同时完成实时全身捕捉。本研究提出一种顺序代理到运动学习框架，并构建包含二维骨架序列与三维旋转运动的代理数据集。该代理数据使我们能够构建具备精确全身监督的学习型网络，同时缓解泛化性问题。为实现更精准且物理合理的预测，我们在网络中引入接触感知神经运动降维模块，使其能够感知脚地接触状态与代理观测间的运动偏移。此外，我们通过共享全身-手部上下文信息，实现与全身模型更兼容的手腕姿态恢复。基于所提出的学习方案，我们首次展示了具备世界空间脚地接触合理性且支持实时运算的单目全身捕捉系统。更多视频结果请访问项目主页：https://liuyebin.com/proxycap