We present Score-Guided Human Mesh Recovery (ScoreHMR), an approach for solving inverse problems for 3D human pose and shape reconstruction. These inverse problems involve fitting a human body model to image observations, traditionally solved through optimization techniques. ScoreHMR mimics model fitting approaches, but alignment with the image observation is achieved through score guidance in the latent space of a diffusion model. The diffusion model is trained to capture the conditional distribution of the human model parameters given an input image. By guiding its denoising process with a task-specific score, ScoreHMR effectively solves inverse problems for various applications without the need for retraining the task-agnostic diffusion model. We evaluate our approach on three settings/applications. These are: (i) single-frame model fitting; (ii) reconstruction from multiple uncalibrated views; (iii) reconstructing humans in video sequences. ScoreHMR consistently outperforms all optimization baselines on popular benchmarks across all settings. We make our code and models available at the https://statho.github.io/ScoreHMR.

翻译：我们提出了分数引导人体网格恢复（ScoreHMR），一种解决三维人体姿态与形状重建逆问题的方法。这些逆问题涉及将人体模型拟合到图像观测中，传统上通过优化技术求解。ScoreHMR模拟模型拟合方法，但通过与图像观测的对齐通过扩散模型潜在空间中的分数引导实现。扩散模型被训练捕捉给定输入图像条件下人体模型参数的条件分布。通过使用任务特定分数引导其去噪过程，ScoreHMR有效解决了各种应用的逆问题，无需重新训练任务无关的扩散模型。我们在三个设置/应用上评估了方法：（i）单帧模型拟合；（ii）从多个未校准视图重建；（iii）视频序列中的人体重建。在所有设置的流行基准上，ScoreHMR持续优于所有优化基线。我们在https://statho.github.io/ScoreHMR提供代码和模型。