Estimating 3D humans from images often produces implausible bodies that lean, float, or penetrate the floor. Such methods ignore the fact that bodies are typically supported by the scene. A physics engine can be used to enforce physical plausibility, but these are not differentiable, rely on unrealistic proxy bodies, and are difficult to integrate into existing optimization and learning frameworks. In contrast, we exploit novel intuitive-physics (IP) terms that can be inferred from a 3D SMPL body interacting with the scene. Inspired by biomechanics, we infer the pressure heatmap on the body, the Center of Pressure (CoP) from the heatmap, and the SMPL body's Center of Mass (CoM). With these, we develop IPMAN, to estimate a 3D body from a color image in a "stable" configuration by encouraging plausible floor contact and overlapping CoP and CoM. Our IP terms are intuitive, easy to implement, fast to compute, differentiable, and can be integrated into existing optimization and regression methods. We evaluate IPMAN on standard datasets and MoYo, a new dataset with synchronized multi-view images, ground-truth 3D bodies with complex poses, body-floor contact, CoM and pressure. IPMAN produces more plausible results than the state of the art, improving accuracy for static poses, while not hurting dynamic ones. Code and data are available for research at https://ipman.is.tue.mpg.de.
翻译:从图像中估计三维人体时,常常产生不合理的姿态(如倾斜、悬浮或穿透地面)。此类方法忽略了人体通常受场景支撑这一事实。物理引擎可用于保证物理合理性,但其不可微分、依赖不现实的代理人体,且难以融入现有优化与学习框架。相反,我们利用新颖的直观物理(IP)项,这些项可通过与场景交互的三维SMPL人体推断得到。受生物力学启发,我们推断人体压力热力图、基于热力图提取压力中心(CoP),以及SMPL人体质心(CoM)。基于此,我们开发了IPMAN方法,通过鼓励合理的足地接触以及CoP与CoM的重叠,从彩色图像中估计“稳定”姿态的三维人体。我们的IP项具有直观性、易实现、计算快速、可微分,并能融入现有优化与回归方法。我们在标准数据集和MoYo(含同步多视角图像、复杂姿态真实三维人体、足地接触、质心与压力的新数据集)上评估IPMAN。IPMAN比现有技术产生更合理的结果,在静态姿态上提升精度,同时不损害动态姿态性能。代码与数据已在https://ipman.is.tue.mpg.de公开供研究使用。