In this paper, we introduce FitMe, a facial reflectance model and a differentiable rendering optimization pipeline, that can be used to acquire high-fidelity renderable human avatars from single or multiple images. The model consists of a multi-modal style-based generator, that captures facial appearance in terms of diffuse and specular reflectance, and a PCA-based shape model. We employ a fast differentiable rendering process that can be used in an optimization pipeline, while also achieving photorealistic facial shading. Our optimization process accurately captures both the facial reflectance and shape in high-detail, by exploiting the expressivity of the style-based latent representation and of our shape model. FitMe achieves state-of-the-art reflectance acquisition and identity preservation on single "in-the-wild" facial images, while it produces impressive scan-like results, when given multiple unconstrained facial images pertaining to the same identity. In contrast with recent implicit avatar reconstructions, FitMe requires only one minute and produces relightable mesh and texture-based avatars, that can be used by end-user applications.

翻译：本文提出FitMe，一种面部反射模型及可微渲染优化流程，可通过单张或多张图像获取高保真可渲染人体化身。该模型由多模态风格生成器（用于捕捉以漫反射和镜面反射表示的面部外观）与基于主成分分析（PCA）的形状模型构成。我们采用快速可微渲染流程用于优化管线，同时实现照片级面部着色。通过利用风格化潜在表示与形状模型的表达力，我们的优化流程能够高细节地精确捕捉面部反射与形状。针对单张“野外”面部图像，FitMe在反射率获取与身份保留方面达到当前最优性能；而当提供同一身份的多张非约束面部图像时，可产生媲美扫描结果的高质量输出。与近期隐式化身重建方法相比，FitMe仅需一分钟即可生成可重光照的网格与纹理化身，可直接应用于终端用户场景。