The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because hair has complex geometry and appearance, as well as exhibits challenging motion. In this paper, we present a two-stage approach that models hair independently from the head to address these challenges in a data-driven manner. The first stage, state compression, learns a low-dimensional latent space of 3D hair states containing motion and appearance, via a novel autoencoder-as-a-tracker strategy. To better disentangle the hair and head in appearance learning, we employ multi-view hair segmentation masks in combination with a differentiable volumetric renderer. The second stage learns a novel hair dynamics model that performs temporal hair transfer based on the discovered latent codes. To enforce higher stability while driving our dynamics model, we employ the 3D point-cloud autoencoder from the compression stage for de-noising of the hair state. Our model outperforms the state of the art in novel view synthesis and is capable of creating novel hair animations without having to rely on hair observations as a driving signal. Project page is here https://ziyanw1.github.io/neuwigs/.

翻译：人类头发的捕捉与动画是虚拟现实中创建逼真化身的两大主要挑战。这两个问题极具挑战性，因为头发具有复杂的几何形状与外观，且表现出复杂的运动。本文提出了一种两阶段方法，通过数据驱动方式将头发与头部独立建模以应对这些挑战。第一阶段——状态压缩，通过一种新颖的"自编码器-追踪器"策略，学习包含运动与外观的3D头发状态的低维潜在空间。为在外观学习中更好地解耦头发与头部，我们结合多视角头发分割掩模与可微分体积渲染器。第二阶段学习了一种新颖的头发动力学模型，该模型基于所发现的潜在编码执行时序头发迁移。为在驱动动力学模型时实现更高稳定性，我们使用压缩阶段的3D点云自编码器对头发状态进行去噪。我们的模型在新视角合成任务中优于现有技术，并且能够无需依赖头发观测作为驱动信号即可生成新颖的头发动画。项目页面位于：https://ziyanw1.github.io/neuwigs/。