We present an approach to reconstruct humans and track them over time. At the core of our approach, we propose a fully "transformerized" version of a network for human mesh recovery. This network, HMR 2.0, advances the state of the art and shows the capability to analyze unusual poses that have in the past been difficult to reconstruct from single images. To analyze video, we use 3D reconstructions from HMR 2.0 as input to a tracking system that operates in 3D. This enables us to deal with multiple people and maintain identities through occlusion events. Our complete approach, 4DHumans, achieves state-of-the-art results for tracking people from monocular video. Furthermore, we demonstrate the effectiveness of HMR 2.0 on the downstream task of action recognition, achieving significant improvements over previous pose-based action recognition approaches. Our code and models are available on the project website: https://shubham-goel.github.io/4dhumans/.

翻译：我们提出一种对人体进行重建并持续追踪的方法。该方法的核心理念在于构建一个完全"Transformer化"的人体网格恢复网络。该网络HMR 2.0不仅推动了该领域技术发展前沿，更展现出分析非常规姿态的能力——这些姿态在以往单目图像重建中往往难以处理。针对视频分析环节，我们以HMR 2.0输出的三维重建结果作为三维追踪系统的输入，通过这种空间维度运作机制，有效应对多人场景中的人体身份保持与遮挡处理问题。我们提出的完整方案——4DHumans——在单目视频人体追踪领域取得了领先成果。此外，我们在动作识别下游任务中验证了HMR 2.0的卓越性能，相较于传统基于姿态的动作识别方法实现了显著提升。相关代码与模型已开源于项目网站：https://shubham-goel.github.io/4dhumans/。