This work studies learning from a synergy process of 3D Morphable Models (3DMM) and 3D facial landmarks to predict complete 3D facial geometry, including 3D alignment, face orientation, and 3D face modeling. Our synergy process leverages a representation cycle for 3DMM parameters and 3D landmarks. 3D landmarks can be extracted and refined from face meshes built by 3DMM parameters. We next reverse the representation direction and show that predicting 3DMM parameters from sparse 3D landmarks improves the information flow. Together we create a synergy process that utilizes the relation between 3D landmarks and 3DMM parameters, and they collaboratively contribute to better performance. We extensively validate our contribution on full tasks of facial geometry prediction and show our superior and robust performance on these tasks for various scenarios. Particularly, we adopt only simple and widely-used network operations to attain fast and accurate facial geometry prediction. Codes and data: https://choyingw.github.io/works/SynergyNet/

翻译：本文研究利用三维形变模型（3DMM）与三维人脸关键点的协同学习过程来预测完整的三维人脸几何结构，包括三维对齐、人脸朝向及三维人脸建模。所提出的协同过程基于3DMM参数与三维关键点的表征循环机制：首先从3DMM参数构建的人脸网格中提取并优化三维关键点；随后逆转表征方向，证明从稀疏三维关键点预测3DMM参数能够改善信息流。通过联合构建的协同过程，我们有效利用了三维关键点与3DMM参数之间的关联性，促使两者共同提升模型性能。本文在人脸几何预测的全部任务上进行了充分验证，结果表明该方法在多种场景下均具有卓越的鲁棒性。特别值得注意的是，我们仅采用简单且广泛应用的网络操作，即可实现快速准确的三维人脸几何预测。代码与数据开源地址：https://choyingw.github.io/works/SynergyNet/