While 3D body reconstruction methods have made remarkable progress recently, it remains difficult to acquire the sufficiently accurate and numerous 3D supervisions required for training. In this paper, we propose \textbf{KNOWN}, a framework that effectively utilizes body \textbf{KNOW}ledge and u\textbf{N}certainty modeling to compensate for insufficient 3D supervisions. KNOWN exploits a comprehensive set of generic body constraints derived from well-established body knowledge. These generic constraints precisely and explicitly characterize the reconstruction plausibility and enable 3D reconstruction models to be trained without any 3D data. Moreover, existing methods typically use images from multiple datasets during training, which can result in data noise (\textit{e.g.}, inconsistent joint annotation) and data imbalance (\textit{e.g.}, minority images representing unusual poses or captured from challenging camera views). KNOWN solves these problems through a novel probabilistic framework that models both aleatoric and epistemic uncertainty. Aleatoric uncertainty is encoded in a robust Negative Log-Likelihood (NLL) training loss, while epistemic uncertainty is used to guide model refinement. Experiments demonstrate that KNOWN's body reconstruction outperforms prior weakly-supervised approaches, particularly on the challenging minority images.
翻译:尽管近年来三维人体重建方法取得了显著进展,但在训练过程中获取足够精确且数量充足的三维监督数据仍然困难。本文提出KNOWN框架,通过有效利用人体知识与不确定性建模来弥补三维监督数据的不足。KNOWN利用源自成熟人体知识的一系列通用人体约束,这些通用约束能够精确、显式地表征重建合理性,使得无需任何三维数据即可训练三维重建模型。此外,现有方法在训练时通常使用来自多个数据集的图像,这会导致数据噪声(例如关节点标注不一致)和数据不平衡(例如代表特殊姿态或从具有挑战性的相机视角拍摄的少数图像)。KNOWN通过一种新颖的概率框架建模偶然不确定性与认知不确定性来解决这些问题。偶然不确定性被编码为鲁棒的负对数似然(NLL)训练损失,而认知不确定性则用于引导模型优化。实验证明,KNOWN的人体重建效果优于先前的弱监督方法,尤其在对挑战性的少数图像处理上表现突出。