Current advances in human head modeling allow to generate plausible-looking 3D head models via neural representations. Nevertheless, constructing complete high-fidelity head models with explicitly controlled animation remains an issue. Furthermore, completing the head geometry based on a partial observation, e.g. coming from a depth sensor, while preserving details is often problematic for the existing methods. We introduce a generative model for detailed 3D head meshes on top of an articulated 3DMM which allows explicit animation and high-detail preservation at the same time. Our method is trained in two stages. First, we register a parametric head model with vertex displacements to each mesh of the recently introduced NPHM dataset of accurate 3D head scans. The estimated displacements are baked into a hand-crafted UV layout. Second, we train a StyleGAN model in order to generalize over the UV maps of displacements. The decomposition of the parametric model and high-quality vertex displacements allows us to animate the model and modify it semantically. We demonstrate the results of unconditional generation and fitting to the full or partial observation. The project page is available at https://seva100.github.io/headcraft.

翻译：当前人类头部建模领域的最新进展可通过神经表征生成外观逼真的3D头部模型。然而，构建具有明确动画控制的完整高保真头部模型仍存在挑战。此外，现有方法在面对基于部分观测（例如来自深度传感器的数据）完成头部几何重建并保留细节时，往往难以处理。我们提出了一种生成模型，在带关节的3DMM基础上构建细节丰富的3D头部网格，该模型同时支持显式动画与高细节保留。我们的方法分两个阶段训练：首先，将带有顶点位移的参数化头部模型配准到最新发布的NPHM精准3D头部扫描数据集的每个网格上，并将估计的位移映射至手动设计的UV布局中；其次，训练StyleGAN模型以实现对位移UV图的泛化。参数化模型与高质量顶点位移的分解机制，使我们能够驱动模型动画并执行语义编辑。我们展示了无条件生成以及适配完整/部分观测数据的结果。项目页面详见https://seva100.github.io/headcraft。