Self-supervised facial representation has recently attracted increasing attention due to its ability to perform face understanding without relying on large-scale annotated datasets heavily. However, analytically, current contrastive-based self-supervised learning (SSL) still performs unsatisfactorily for learning facial representation. More specifically, existing contrastive learning (CL) tends to learn pose-invariant features that cannot depict the pose details of faces, compromising the learning performance. To conquer the above limitation of CL, we propose a novel Pose-disentangled Contrastive Learning (PCL) method for general self-supervised facial representation. Our PCL first devises a pose-disentangled decoder (PDD) with a delicately designed orthogonalizing regulation, which disentangles the pose-related features from the face-aware features; therefore, pose-related and other pose-unrelated facial information could be performed in individual subnetworks and do not affect each other's training. Furthermore, we introduce a pose-related contrastive learning scheme that learns pose-related information based on data augmentation of the same image, which would deliver more effective face-aware representation for various downstream tasks. We conducted linear evaluation on four challenging downstream facial understanding tasks, ie, facial expression recognition, face recognition, AU detection and head pose estimation. Experimental results demonstrate that our method significantly outperforms state-of-the-art SSL methods. Code is available at https://github.com/DreamMr/PCL}{https://github.com/DreamMr/PCL
翻译:自监督面部表示近年来因能够在无需依赖大规模标注数据集的情况下进行面部理解而受到越来越多的关注。然而,分析表明,当前的基于对比的自监督学习方法在学习面部表示方面仍表现不佳。具体而言,现有的对比学习倾向于学习姿态不变的特征,这些特征无法刻画面部的姿态细节,从而影响了学习性能。为解决上述对比学习的局限性,我们提出了一种新颖的姿态解耦对比学习方法,用于通用的自监督面部表示。我们的方法首先设计了一个姿态解耦解码器,并带有精心设计的正交化正则化项,该解码器将姿态相关特征与人脸感知特征分离;因此,姿态相关和其他与姿态无关的面部信息可以在单独的子网络中进行处理,且互不影响训练。此外,我们引入了一种姿态相关的对比学习方案,该方案基于同一图像的数据增强来学习姿态相关信息,从而为各种下游任务提供更有效的人脸感知表示。我们在四个具有挑战性的下游面部理解任务(即面部表情识别、人脸识别、动作单元检测和头部姿态估计)上进行了线性评估。实验结果表明,我们的方法显著优于最先进的自监督学习方法。代码可在 https://github.com/DreamMr/PCL 获取。