Human mesh recovery (HMR) provides rich human body information for various real-world applications such as gaming, human-computer interaction, and virtual reality. Compared to single image-based methods, video-based methods can utilize temporal information to further improve performance by incorporating human body motion priors. However, many-to-many approaches such as VIBE suffer from motion smoothness and temporal inconsistency. While many-to-one approaches such as TCMR and MPS-Net rely on the future frames, which is non-causal and time inefficient during inference. To address these challenges, a novel Diffusion-Driven Transformer-based framework (DDT) for video-based HMR is presented. DDT is designed to decode specific motion patterns from the input sequence, enhancing motion smoothness and temporal consistency. As a many-to-many approach, the decoder of our DDT outputs the human mesh of all the frames, making DDT more viable for real-world applications where time efficiency is crucial and a causal model is desired. Extensive experiments are conducted on the widely used datasets (Human3.6M, MPI-INF-3DHP, and 3DPW), which demonstrated the effectiveness and efficiency of our DDT.

翻译：人体网格恢复（Human Mesh Recovery, HMR）为游戏、人机交互和虚拟现实等实际应用提供了丰富的人体信息。与基于单张图像的方法相比，基于视频的方法可通过融合人体运动先验信息，利用时序信息进一步提升性能。然而，VIBE等多对多方法存在运动平滑度不足和时序不一致问题；而TCMR和MPS-Net等多对一方法依赖未来帧，在推理过程中存在非因果性和时间效率低下的缺陷。为解决上述挑战，本文提出了一种新颖的基于扩散驱动Transformer的视频HMR框架（DDT）。DDT设计用于从输入序列中解码特定运动模式，增强运动平滑度与时序一致性。作为一种多对多方法，DDT的解码器可输出所有帧的人体网格，使其在需要高效率以及因果模型的现实应用中更具可行性。在广泛使用的数据集（Human3.6M、MPI-INF-3DHP和3DPW）上进行的大量实验证明了DDT的有效性与效率。