This work proposes a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. The existing models, which are simplified joint-actuated systems, often produce unnatural motions. To address this, we integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create a new model, MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR is conducted in two separate parts. The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale public datasets and two recent state-of-the-art methods. The results demonstrate that our approach consistently improves the baseline methods both quantitatively and qualitatively.

翻译：本文提出了一种新颖的学习框架，用于视觉手部动力学分析，该框架充分考虑了手部运动的生理学特性。现有的简化关节驱动系统模型常常产生非自然的运动。为解决这一问题，我们整合了一个肌肉骨骼系统与可学习的参数化手部模型MANO，构建了新模型MS-MANO。该模型模拟肌肉和肌腱的动力学特性以驱动骨骼系统，对生成的扭矩轨迹施加了符合生理实际的约束。我们进一步提出了一种仿真在环姿态细化框架BioPR，通过多层感知器（MLP）网络对初始估计姿态进行优化。针对MS-MANO的精度与BioPR的有效性，我们分两部分进行了评估。MS-MANO的精度与MyoSuite进行了比较，而BioPR的有效性则基于两个大规模公开数据集及两种最新最先进方法进行了基准测试。结果表明，我们的方法在定量和定性两个层面均持续优于基线方法。