Facial makeup enriches the beauty of not only real humans but also virtual characters; therefore, makeup for 3D facial models is highly in demand in productions. However, painting directly on 3D faces and capturing real-world makeup are costly, and extracting makeup from 2D images often struggles with shading effects and occlusions. This paper presents the first method for extracting makeup for 3D facial models from a single makeup portrait. Our method consists of the following three steps. First, we exploit the strong prior of 3D morphable models via regression-based inverse rendering to extract coarse materials such as geometry and diffuse/specular albedos that are represented in the UV space. Second, we refine the coarse materials, which may have missing pixels due to occlusions. We apply inpainting and optimization. Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse albedo. Our method offers various applications for not only 3D facial models but also 2D portrait images. The extracted makeup is well-aligned in the UV space, from which we build a large-scale makeup dataset and a parametric makeup model for 3D faces. Our disentangled materials also yield robust makeup transfer and illumination-aware makeup interpolation/removal without a reference image.

翻译：面部妆容不仅为真实人类增添美感，也常用于虚拟角色；因此，3D面部模型的妆容在影视制作中需求极高。然而，直接在3D面部绘制妆容或捕捉真实世界妆容成本高昂，而从2D图像中提取妆容常面临阴影效果和遮挡问题。本文提出首个从单张肖像图像中提取3D面部模型妆容的方法。该方法包含以下三个步骤：首先，利用回归式逆渲染技术结合3D可变形模型的强先验知识，提取以UV空间表示的几何与漫反射/镜面反射反照率等粗略材质信息；其次，针对因遮挡导致的像素缺失问题，通过补全与优化算法对粗略材质进行精细化处理；最后，从漫反射反照率中分离出裸肤、妆容及alpha遮罩。本方法不仅适用于3D面部模型，还可拓展至2D肖像图像的多项应用。提取的妆容在UV空间中完美对齐，由此构建了大规模妆容数据集及面向3D面部的参数化妆容模型。此外，解耦后的材质参数无需参照图像即可实现鲁棒的妆容迁移与光照感知的妆容插值/去除。