We present an approach to reconstruct humans and track them over time. At the core of our approach, we propose a fully "transformerized" version of a network for human mesh recovery. This network, HMR 2.0, advances the state of the art and shows the capability to analyze unusual poses that have in the past been difficult to reconstruct from single images. To analyze video, we use 3D reconstructions from HMR 2.0 as input to a tracking system that operates in 3D. This enables us to deal with multiple people and maintain identities through occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art results for tracking people from monocular video. Furthermore, we demonstrate the effectiveness of HMR 2.0 on the downstream task of action recognition, achieving significant improvements over previous pose-based action recognition approaches. Our code and models are available on the project website: https://shubham-goel.github.io/4dhumans/.

翻译：我们提出了一种对人体进行重构并随时间跟踪的方法。该方法的核心是提出一个完全"Transformer化"的人体网格恢复网络架构。该网络HMR 2.0推动了技术前沿发展，展现出分析非常规姿态的能力——这类姿态在过去难以通过单张图像进行重构。为了分析视频数据，我们采用HMR 2.0的三维重构结果作为三维空间跟踪系统的输入，从而能够处理多人场景并在遮挡事件中维持身份一致性。我们的完整方案4DHumans在单目视频人体跟踪任务上取得了最优结果。此外，我们在动作识别下游任务中验证了HMR 2.0的有效性，相比以往基于姿态的动作识别方法实现了显著提升。相关代码与模型已发布于项目网站：https://shubham-goel.github.io/4dhumans/。