Multi-view volumetric rendering techniques have recently shown great potential in modeling and synthesizing high-quality head avatars. A common approach to capture full head dynamic performances is to track the underlying geometry using a mesh-based template or 3D cube-based graphics primitives. While these model-based approaches achieve promising results, they often fail to learn complex geometric details such as the mouth interior, hair, and topological changes over time. This paper presents a novel approach to building highly photorealistic digital head avatars. Our method learns a canonical space via an implicit function parameterized by a neural network. It leverages multiresolution hash encoding in the learned feature space, allowing for high-quality, faster training and high-resolution rendering. At test time, our method is driven by a monocular RGB video. Here, an image encoder extracts face-specific features that also condition the learnable canonical space. This encourages deformation-dependent texture variations during training. We also propose a novel optical flow based loss that ensures correspondences in the learned canonical space, thus encouraging artifact-free and temporally consistent renderings. We show results on challenging facial expressions and show free-viewpoint renderings at interactive real-time rates for medium image resolutions. Our method outperforms all existing approaches, both visually and numerically. We will release our multiple-identity dataset to encourage further research. Our Project page is available at: https://vcai.mpi-inf.mpg.de/projects/HQ3DAvatar/

翻译：多视角体积渲染技术近年来在建模和合成高质量头部化身方面展现出巨大潜力。捕获完整头部动态表现的常用方法是利用基于网格模板或三维立方体图形基元追踪底层几何结构。虽然这类基于模型的方法取得了显著成果，但往往难以学习复杂的几何细节，如口腔内部、毛发以及随时间变化的拓扑结构。本文提出了一种构建高逼真度数字头部化身的新方法。该方法通过神经网络参数化的隐式函数学习规范空间，并在学习到的特征空间中利用多分辨率哈希编码，实现了高质量、更快速训练和高分辨率渲染。在测试阶段，我们的方法由单目RGB视频驱动。其中，图像编码器提取人脸特定特征，这些特征同时约束可学习的规范空间，从而在训练过程中鼓励依赖形变的纹理变化。我们还提出了一种基于光流的新损失函数，确保学习到的规范空间中的对应关系，从而产生无伪影且时间一致的渲染结果。我们展示了具有挑战性的面部表情结果，并以中等图像分辨率实现了交互式实时自由视角渲染。我们的方法在视觉和数值上均优于所有现有方法。我们将发布多身份数据集以推动进一步研究。项目页面详见：https://vcai.mpi-inf.mpg.de/projects/HQ3DAvatar/