In high-end visual effects pipelines, a customized (and expensive) light stage system is (typically) used to scan an actor in order to acquire both geometry and texture for various expressions. Aiming towards democratization, we propose a novel pipeline for obtaining geometry and texture as well as enough expression information to build a customized person-specific animation rig without using a light stage or any other high-end hardware (or manual cleanup). A key novel idea consists of warping real-world images to align with the geometry of a template avatar and subsequently projecting the warped image into the template avatar's texture; importantly, this allows us to leverage baked-in real-world lighting/texture information in order to create surrogate facial features (and bridge the domain gap) for the sake of geometry reconstruction. Not only can our method be used to obtain a neutral expression geometry and de-lit texture, but it can also be used to improve avatars after they have been imported into an animation system (noting that such imports tend to be lossy, while also hallucinating various features). Since a default animation rig will contain template expressions that do not correctly correspond to those of a particular individual, we use a Simon Says approach to capture various expressions and build a person-specific animation rig (that moves like they do). Our aforementioned warping/projection method has high enough efficacy to reconstruct geometry corresponding to each expressions.

翻译：在高端视觉特效流程中，通常需要定制化（且昂贵）的光学舞台系统扫描演员，以获取不同表情下的几何体与纹理。为推进技术民主化，我们提出一种新型流水线，无需使用光学舞台或其他高端硬件（及人工清理），即可获取几何体、纹理及充足的表情信息，从而构建个性化动画绑定模型。核心创新在于：通过扭曲真实世界图像使其与模板头像几何体对齐，进而将扭曲图像投影至模板头像纹理。这一方法可有效利用嵌入的真实光照/纹理信息生成替代性面部特征（并弥合领域差异），用于几何重建。我们的方法不仅能够获取中性表情几何体与去光照纹理，还能在头像导入动画系统后提升其质量（需注意此类导入过程通常有损，且易产生伪影）。由于默认动画绑定包含的模板表情与特定个体的实际表情存在偏差，我们采用"西蒙说"方法捕捉多样化表情，构建具有个性化运动特征的动画绑定模型。前述扭曲/投影方法具有足够高的效能，可重建对应于每个表情的几何结构。