Estimating 3D humans from images often produces implausible bodies that lean, float, or penetrate the floor. Such methods ignore the fact that bodies are typically supported by the scene. A physics engine can be used to enforce physical plausibility, but these are not differentiable, rely on unrealistic proxy bodies, and are difficult to integrate into existing optimization and learning frameworks. In contrast, we exploit novel intuitive-physics (IP) terms that can be inferred from a 3D SMPL body interacting with the scene. Inspired by biomechanics, we infer the pressure heatmap on the body, the Center of Pressure (CoP) from the heatmap, and the SMPL body's Center of Mass (CoM). With these, we develop IPMAN, to estimate a 3D body from a color image in a "stable" configuration by encouraging plausible floor contact and overlapping CoP and CoM. Our IP terms are intuitive, easy to implement, fast to compute, differentiable, and can be integrated into existing optimization and regression methods. We evaluate IPMAN on standard datasets and MoYo, a new dataset with synchronized multi-view images, ground-truth 3D bodies with complex poses, body-floor contact, CoM and pressure. IPMAN produces more plausible results than the state of the art, improving accuracy for static poses, while not hurting dynamic ones. Code and data are available for research at https://ipman.is.tue.mpg.de.
翻译:从图像中估计三维人体时,常会产生倾斜、悬浮或穿透地面的不合理姿态。这类方法忽略了人体通常受场景支撑的事实。物理引擎虽可用于确保物理合理性,但其不可微分、依赖非真实代理体,且难以集成至现有优化与学习框架中。与此相反,我们利用与场景交互的三维SMPL人体推断新型直觉物理(IP)项。受生物力学启发,我们推断人体压力热力图、从热力图提取的压力中心(CoP)以及SMPL人体质心(CoM)。基于此,我们开发IPMAN方法,通过鼓励合理的地面接触及CoP与CoM重叠,从彩色图像中估计"稳定"配置的三维人体。我们的IP项直观易实现、计算快速、可微分,并能集成至现有优化与回归方法中。我们在标准数据集及MoYo(包含同步多视角图像、具有复杂姿态的真实三维人体、人体-地面接触、CoM与压力信息的新数据集)上评估IPMAN。该方法较现有技术产生更合理的结果,在静态姿态上提升准确性且不影响动态姿态。研究相关代码与数据可于https://ipman.is.tue.mpg.de获取。