Estimating 3D humans from images often produces implausible bodies that lean, float, or penetrate the floor. Such methods ignore the fact that bodies are typically supported by the scene. A physics engine can be used to enforce physical plausibility, but these are not differentiable, rely on unrealistic proxy bodies, and are difficult to integrate into existing optimization and learning frameworks. In contrast, we exploit novel intuitive-physics (IP) terms that can be inferred from a 3D SMPL body interacting with the scene. Inspired by biomechanics, we infer the pressure heatmap on the body, the Center of Pressure (CoP) from the heatmap, and the SMPL body's Center of Mass (CoM). With these, we develop IPMAN, to estimate a 3D body from a color image in a "stable" configuration by encouraging plausible floor contact and overlapping CoP and CoM. Our IP terms are intuitive, easy to implement, fast to compute, differentiable, and can be integrated into existing optimization and regression methods. We evaluate IPMAN on standard datasets and MoYo, a new dataset with synchronized multi-view images, ground-truth 3D bodies with complex poses, body-floor contact, CoM and pressure. IPMAN produces more plausible results than the state of the art, improving accuracy for static poses, while not hurting dynamic ones. Code and data are available for research at https://ipman.is.tue.mpg.de.
翻译:从图像中估计3D人体常会生成不合理的身体姿态,例如身体倾斜、悬浮或穿透地面。这类方法忽略了人体通常受场景支撑这一事实。物理引擎虽可用于增强物理合理性,但其不可微分、依赖不切实际的代理体,且难以集成到现有优化与学习框架中。相反,我们利用新型直观物理(IP)项——这些项可通过与场景交互的3D SMPL人体推断得出。受生物力学启发,我们从人体压力热力图推断压力中心(CoP),并结合SMPL人体的质心(CoM)。基于此,我们开发了IPMAN方法,通过鼓励合理的足地接触与CoP-CoM重叠,从彩色图像中估计"稳定"姿态下的3D人体。所提IP项直观易实现、计算快速、可微分,并能集成到现有优化与回归方法中。我们在标准数据集及MoYo(包含同步多视角图像、带复杂姿态的真实3D人体、足地接触、CoM与压力数据的新数据集)上评估IPMAN。与现有最优方法相比,IPMAN在生成更合理结果的同时,提升了静态姿态的精度,且未损害动态姿态性能。代码与数据已公开供研究使用:https://ipman.is.tue.mpg.de。