To reconstruct a 3D human surface from a single image, it is important to consider human pose, shape and clothing details simultaneously. In recent years, a combination of parametric body models (such as SMPL) that capture body pose and shape prior, and neural implicit functions that learn flexible clothing details, has been used to integrate the advantages of both approaches. However, the combined representation introduces additional computation, e.g. signed distance calculation, in 3D body feature extraction, which exacerbates the redundancy of the implicit query-and-infer process and fails to preserve the underlying body shape prior. To address these issues, we propose a novel IUVD-Feedback representation, which consists of an IUVD occupancy function and a feedback query algorithm. With this representation, the time-consuming signed distance calculation is replaced by a simple linear transformation in the IUVD space, leveraging the SMPL UV maps. Additionally, the redundant query points in the query-and-infer process are reduced through a feedback mechanism. This leads to more reasonable 3D body features and more effective query points, successfully preserving the parametric body prior. Moreover, the IUVD-Feedback representation can be embedded into any existing implicit human reconstruction pipelines without modifying the trained neural networks. Experiments on THuman2.0 dataset demonstrate that the proposed IUVD-Feedback representation improves result robustness and achieves three times faster acceleration in the query-and-infer process. Furthermore, this representation has the potential to be used in generative applications by leveraging its inherited semantic information from the parametric body model.
翻译:从单张图像重建三维人体表面时,需同步考虑人体姿态、体形及衣物细节。近年来,结合参数化人体模型(如SMPL)捕捉姿态与体形先验信息,以及利用神经隐式函数学习灵活衣物细节的方法,已用于融合两类技术的优势。然而,这种组合表示在三维人体特征提取过程中引入了额外计算(如符号距离计算),加剧了隐式查询-推理流程的冗余性,且无法保留底层体形先验。针对这些问题,我们提出一种新颖的IUVD-反馈表示,其包含IUVD占据函数与反馈查询算法。通过该表示,利用SMPL UV映射可在IUVD空间中以简单线性变换替代耗时的符号距离计算。此外,通过反馈机制减少了查询-推理流程中的冗余查询点,从而获得更合理的三维人体特征与更有效的查询点,成功保留参数化人体先验。值得注意的是,IUVD-反馈表示可嵌入任何现有隐式人体重建流程中,而无需修改已训练的神经网络。在THuman2.0数据集上的实验表明,所提IUVD-反馈表示能提升结果鲁棒性,并在查询-推理流程中实现三倍加速。此外,该表示可利用从参数化人体模型中继承的语义信息,具有用于生成式应用的潜力。