This paper addresses the challenging task of reconstructing the poses of multiple individuals engaged in close interactions, captured by multiple calibrated cameras. The difficulty arises from the noisy or false 2D keypoint detections due to inter-person occlusion, the heavy ambiguity in associating keypoints to individuals due to the close interactions, and the scarcity of training data as collecting and annotating motion data in crowded scenes is resource-intensive. We introduce a novel system to address these challenges. Our system integrates a learning-based pose estimation component and its corresponding training and inference strategies. The pose estimation component takes multi-view 2D keypoint heatmaps as input and reconstructs the pose of each individual using a 3D conditional volumetric network. As the network doesn't need images as input, we can leverage known camera parameters from test scenes and a large quantity of existing motion capture data to synthesize massive training data that mimics the real data distribution in test scenes. Extensive experiments demonstrate that our approach significantly surpasses previous approaches in terms of pose accuracy and is generalizable across various camera setups and population sizes. The code is available on our project page: https://github.com/zju3dv/CloseMoCap.

翻译：本文针对多个个体在密切交互场景中的姿态重建这一挑战性问题进行研究，此类场景由多台标定相机捕获。其难点源于人际遮挡导致的二维关键点检测噪声或错误、密切交互引发的关键点与个体关联的高度模糊性，以及因拥挤场景中运动数据采集与标注资源密集而导致的训练数据匮乏。我们提出了一种新颖系统以应对这些挑战。该系统集成了基于学习的姿态估计组件及其对应的训练与推理策略。姿态估计组件以多视角二维关键点热图为输入，通过三维条件体积网络重建每个个体的姿态。由于该网络无需图像作为输入，我们可利用测试场景的已知相机参数与大量现有动作捕捉数据，合成模拟测试场景真实数据分布的海量训练数据。大量实验表明，我们的方法在姿态精度上显著超越先前方法，并能泛化至多种相机设置与人群规模。代码已发布于项目主页：https://github.com/zju3dv/CloseMoCap。