Human mesh recovery (HMR) provides rich human body information for various real-world applications such as gaming, human-computer interaction, and virtual reality. Compared to single image-based methods, video-based methods can utilize temporal information to further improve performance by incorporating human body motion priors. However, many-to-many approaches such as VIBE suffer from motion smoothness and temporal inconsistency. While many-to-one approaches such as TCMR and MPS-Net rely on the future frames, which is non-causal and time inefficient during inference. To address these challenges, a novel Diffusion-Driven Transformer-based framework (DDT) for video-based HMR is presented. DDT is designed to decode specific motion patterns from the input sequence, enhancing motion smoothness and temporal consistency. As a many-to-many approach, the decoder of our DDT outputs the human mesh of all the frames, making DDT more viable for real-world applications where time efficiency is crucial and a causal model is desired. Extensive experiments are conducted on the widely used datasets (Human3.6M, MPI-INF-3DHP, and 3DPW), which demonstrated the effectiveness and efficiency of our DDT.

翻译：人体网格恢复（HMR）为游戏、人机交互和虚拟现实等实际应用提供了丰富的人体信息。与基于单图像的方法相比，基于视频的方法可以利用时间信息，通过融合人体运动先验来进一步提升性能。然而，诸如VIBE等“多对多”方法存在运动平滑性和时间一致性问题，而TCMR和MPS-Net等“多对一”方法依赖于未来帧，这在推理过程中既非因果也耗时低效。为解决这些挑战，本文提出了一种新颖的基于扩散驱动的Transformer框架（DDT），用于视频HMR。DDT旨在从输入序列中解码特定运动模式，增强运动平滑性和时间一致性。作为一种“多对多”方法，DDT的解码器输出所有帧的人体网格，使其更适用于对时间效率要求高且需因果模型的现实应用。在广泛使用的数据集（Human3.6M、MPI-INF-3DHP和3DPW）上进行了大量实验，结果证明了DDT的有效性和高效性。