Extracting physically plausible 3D human motion from videos is a critical task. Although existing simulation-based motion imitation methods can enhance the physical quality of daily motions estimated from monocular video capture, extending this capability to high-difficulty motions remains an open challenge. This can be attributed to some flawed motion clips in video-based motion capture results and the inherent complexity in modeling high-difficulty motions. Therefore, sensing the advantage of segmentation in localizing human body, we introduce a mask-based motion correction module (MCM) that leverages motion context and video mask to repair flawed motions, producing imitation-friendly motions; and propose a physics-based motion transfer module (PTM), which employs a pretrain and adapt approach for motion imitation, improving physical plausibility with the ability to handle in-the-wild and challenging motions. Our approach is designed as a plug-and-play module to physically refine the video motion capture results, including high-difficulty in-the-wild motions. Finally, to validate our approach, we collected a challenging in-the-wild test set to establish a benchmark, and our method has demonstrated effectiveness on both the new benchmark and existing public datasets.https://physicalmotionrestoration.github.io

翻译：从视频中提取物理上合理的三维人体运动是一项关键任务。尽管现有的基于仿真的运动模仿方法能够提升从单目视频捕捉估计的日常运动的物理质量，但将此能力扩展到高难度运动仍是一个开放挑战。这可以归因于基于视频的运动捕捉结果中存在一些有缺陷的运动片段，以及高难度运动建模固有的复杂性。因此，感知到分割在人体定位方面的优势，我们引入了一个基于掩码的运动校正模块（MCM），它利用运动上下文和视频掩码来修复有缺陷的运动，生成易于模仿的运动；并提出一个基于物理的运动迁移模块（PTM），它采用预训练与适配的方法进行运动模仿，通过处理野外及挑战性运动的能力来提升物理合理性。我们的方法被设计为一个即插即用模块，用于物理优化视频运动捕捉结果，包括野外高难度运动。最后，为验证我们的方法，我们收集了一个具有挑战性的野外测试集以建立基准，我们的方法在新的基准和现有的公共数据集上都证明了其有效性。https://physicalmotionrestoration.github.io