The recovery of 3D human mesh from monocular images has significantly been developed in recent years. However, existing models usually ignore spatial and temporal information, which might lead to mesh and image misalignment and temporal discontinuity. For this reason, we propose a novel Spatio-Temporal Alignment Fusion (STAF) model. As a video-based model, it leverages coherence clues from human motion by an attention-based Temporal Coherence Fusion Module (TCFM). As for spatial mesh-alignment evidence, we extract fine-grained local information through predicted mesh projection on the feature maps. Based on the spatial features, we further introduce a multi-stage adjacent Spatial Alignment Fusion Module (SAFM) to enhance the feature representation of the target frame. In addition to the above, we propose an Average Pooling Module (APM) to allow the model to focus on the entire input sequence rather than just the target frame. This method can remarkably improve the smoothness of recovery results from video. Extensive experiments on 3DPW, MPII3D, and H36M demonstrate the superiority of STAF. We achieve a state-of-the-art trade-off between precision and smoothness. Our code and more video results are on the project page https://yw0208.github.io/staf/

翻译：从单目图像恢复三维人体网格在近年来取得了显著进展。然而，现有模型通常忽略空间与时间信息，可能导致网格与图像错位以及时间不连续性问题。为此，我们提出一种新型时空对齐融合（STAF）模型。作为基于视频的模型，它通过基于注意力的时序相干融合模块（TCFM）利用人体运动中的相干线索。针对空间网格对齐证据，我们通过预测网格在特征图上的投影提取细粒度局部信息。基于空间特征，我们进一步引入多阶段相邻空间对齐融合模块（SAFM）以增强目标帧的特征表示。此外，我们提出平均池化模块（APM），使模型能够关注整个输入序列而非仅目标帧。该方法可显著提升视频恢复结果的光滑性。在3DPW、MPII3D和H36M上的大量实验证明了STAF的优越性。我们在精度与光滑性之间实现了最先进的权衡。我们的代码及更多视频结果请访问项目页面 https://yw0208.github.io/staf/