This paper presents a Geometric-Photometric Joint Alignment(GPJA) method, for accurately aligning human expressions by combining geometry and photometric information. Common practices for registering human heads typically involve aligning landmarks with facial template meshes using geometry processing approaches, but often overlook photometric consistency. GPJA overcomes this limitation by leveraging differentiable rendering to align vertices with target expressions, achieving joint alignment in geometry and photometric appearances automatically, without the need for semantic annotation or aligned meshes for training. It features a holistic rendering alignment strategy and a multiscale regularized optimization for robust and fast convergence. The method utilizes derivatives at vertex positions for supervision and employs a gradient-based algorithm which guarantees smoothness and avoids topological defects during the geometry evolution. Experimental results demonstrate faithful alignment under various expressions, surpassing the conventional ICP-based methods and the state-of-the-art deep learning based method. In practical, our method enhances the efficiency of obtaining topology-consistent face models from multi-view stereo facial scanning.

翻译：本文提出了一种几何-光度联合对齐（GPJA）方法，通过融合几何与光度信息实现人体表情的精确对齐。现有的人脸配准方法通常仅依赖几何处理方法通过面部模板网格对齐特征点，但往往忽略光度一致性。GPJA通过利用可微渲染将顶点与目标表情对齐，自动实现几何与光度外观的联合对齐，无需语义标注或对齐网格进行训练。该方法采用全局渲染对齐策略与多尺度正则化优化，实现鲁棒且快速的收敛。算法以顶点位置处的导数为监督信号，并应用基于梯度的优化方法，在几何演化过程中确保平滑性并避免拓扑缺陷。实验结果表明，该方法在各种表情下均能实现可靠对齐，性能超越传统ICP方法及当前最优的深度学习方法。实际应用中，本方法可有效提升从多视角立体人脸扫描中获取拓扑一致人脸模型的效率。