Capturing accurate 3D human pose in the wild would provide valuable data for training pose estimation and motion generation methods. While video-based estimation approaches have become increasingly accurate, they often fail in common scenarios involving self-contact, such as a hand touching the face. In contrast, wearable bioimpedance sensing can cheaply and unobtrusively measure ground-truth skin-to-skin contact. Consequently, we propose a novel framework that combines visual pose estimators with bioimpedance sensing to capture the 3D pose of people by taking self-contact into account. Our method, BioTUCH, initializes the pose using an off-the-shelf estimator and introduces contact-aware pose optimization during measured self-contact: reprojection error and deviations from the input estimate are minimized while enforcing vertex proximity constraints. We validate our approach using a new dataset of synchronized RGB video, bioimpedance measurements, and 3D motion capture. Testing with three input pose estimators, we demonstrate an average of 11.7% improvement in reconstruction accuracy. We also present a miniature wearable bioimpedance sensor that enables efficient large-scale collection of contact-aware training data for improving pose estimation and generation using BioTUCH. Code and data are available at biotuch.is.tue.mpg.de

翻译：在自然场景中捕获精确的3D人体姿态将为姿态估计与运动生成方法的训练提供宝贵数据。尽管基于视频的估计方法已日益精确，但在涉及自接触的常见场景（如手部触碰面部）中仍常失效。相比之下，可穿戴生物阻抗传感能够以低成本、非侵入方式测量真实的皮肤间接触。为此，我们提出一种创新框架，通过结合视觉姿态估计器与生物阻抗传感，在考虑自接触的前提下捕获人体的3D姿态。我们的方法BioTUCH采用现成估计器初始化姿态，并在检测到自接触时引入接触感知的姿态优化：在强制执行顶点邻近约束的同时，最小化重投影误差与输入估计值的偏差。我们通过新构建的同步RGB视频、生物阻抗测量与3D动作捕捉数据集验证了该方法。使用三种输入姿态估计器进行测试，我们证明了重建精度平均提升11.7%。此外，我们展示了一种微型可穿戴生物阻抗传感器，该传感器能够高效大规模采集接触感知训练数据，从而通过BioTUCH提升姿态估计与生成性能。代码与数据详见biotuch.is.tue.mpg.de