Existing Human NeRF methods for reconstructing 3D humans typically rely on multiple 2D images from multi-view cameras or monocular videos captured from fixed camera views. However, in real-world scenarios, human images are often captured from random camera angles, presenting challenges for high-quality 3D human reconstruction. In this paper, we propose SHERF, the first generalizable Human NeRF model for recovering animatable 3D humans from a single input image. SHERF extracts and encodes 3D human representations in canonical space, enabling rendering and animation from free views and poses. To achieve high-fidelity novel view and pose synthesis, the encoded 3D human representations should capture both global appearance and local fine-grained textures. To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding. Global features enhance the information extracted from the single input image and complement the information missing from the partial 2D observation. Point-level features provide strong clues of 3D human structure, while pixel-aligned features preserve more fine-grained details. To effectively integrate the 3D-aware hierarchical feature bank, we design a feature fusion transformer. Extensive experiments on THuman, RenderPeople, ZJU_MoCap, and HuMMan datasets demonstrate that SHERF achieves state-of-the-art performance, with better generalizability for novel view and pose synthesis.

翻译：现有用于三维人体重建的Human NeRF方法通常依赖多视角相机拍摄的多张二维图像或固定视角的单目视频。然而在真实场景中，人体图像往往以随机相机角度拍摄，这对高质量三维人体重建构成挑战。本文提出SHERF，首个从单张输入图像中恢复可驱动三维人体的可泛化Human NeRF模型。SHERF在规范空间中提取并编码三维人体表征，实现自由视角与姿态的渲染与动画。为实现高保真的新视角与新姿态合成，编码的三维人体表征需同时捕捉全局外观与局部精细纹理。为此，我们提出包含全局特征、点级特征和像素对齐特征的三维感知层次特征库以促进信息编码。全局特征增强从单张输入图像中提取的信息，补全二维局部观测缺失的细节；点级特征提供三维人体结构的强线索；像素对齐特征则保留更精细的细节。为有效整合三维感知层次特征库，我们设计了特征融合Transformer。在THuman、RenderPeople、ZJU_MoCap和HuMMan数据集上的大量实验表明，SHERF在新视角与新姿态合成任务中实现了最优性能，并展现出更强的泛化能力。