We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR's expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS, and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR's ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets as well as noisy facial scans in-the-wild, while providing artist-controlled, editable parameters.

翻译：我们提出了一种端到端的深度学习方法，用于野外三维人脸模型的自动绑定与重定向。该方法名为神经面部绑定（NFR），具有三个关键特性：（i）NFR的表情空间保留了可供艺术控制的人类可解释编辑参数；（ii）NFR可直接适用于任意连通性及表情的面部网格；（iii）NFR能够编码并生成任意对象所表现的复杂表情的精细细节。据我们所知，NFR是首个无需手动创建混合形状或对应关系即可提供野外面部网格真实可控形变的方法。我们设计了一个形变自编码器，并通过多数据集训练方案进行训练，该方案充分利用两种数据源的独特优势：一种是基于FACS可解释控制参数的线性三维形变模型（3DMM），另一种是包含精细细节的真实人脸四维捕捉数据。通过多项实验，我们证明了NFR能够在现有广泛数据集以及野外噪声面部扫描中自动生成真实准确的面部形变，同时提供艺术家可控的可编辑参数。