Recent advances in Neural Radiance Fields (NeRF) have demonstrated promising results in 3D scene representations, including 3D human representations. However, these representations often lack crucial information on the underlying human pose and structure, which is crucial for AR/VR applications and games. In this paper, we introduce a novel approach, termed GHNeRF, designed to address these limitations by learning 2D/3D joint locations of human subjects with NeRF representation. GHNeRF uses a pre-trained 2D encoder streamlined to extract essential human features from 2D images, which are then incorporated into the NeRF framework in order to encode human biomechanic features. This allows our network to simultaneously learn biomechanic features, such as joint locations, along with human geometry and texture. To assess the effectiveness of our method, we conduct a comprehensive comparison with state-of-the-art human NeRF techniques and joint estimation algorithms. Our results show that GHNeRF can achieve state-of-the-art results in near real-time.

翻译：神经辐射场（NeRF）的最新进展已在三维场景表示（包括三维人体表示）中展现出令人瞩目的成果。然而，这些表示通常缺乏对人体姿态和结构的关键信息——这对AR/VR应用和游戏至关重要。本文提出一种名为GHNeRF的新方法，旨在通过NeRF表示学习人体对象的二维/三维关节点位置来突破这些局限。GHNeRF利用预训练的二维编码器，从二维图像中高效提取关键人体特征，并将这些特征整合到NeRF框架中以编码人体生物力学信息。这使得我们的网络能够同时学习关节点位置等生物力学特征以及人体几何与纹理。为评估方法有效性，我们与现有最先进的人体NeRF技术和关节点估计算法进行了全面对比。实验结果表明，GHNeRF能够在近实时场景中取得最先进的成果。