Remote Photoplethysmography (rPPG) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised rPPG estimation methods have shown promising potential in estimating rPPG signals from facial regions without relying on ground truth rPPG signals. However, these methods seem oblivious to interference existing in rPPG signals and still result in unsatisfactory performance. In this paper, we propose a novel De-interfered and Descriptive rPPG Estimation Network (DD-rPPGNet) to eliminate the interference within rPPG features for learning genuine rPPG signals. First, we investigate the characteristics of local spatial-temporal similarities of interference and design a novel unsupervised model to estimate the interference. Next, we propose an unsupervised de-interfered method to learn genuine rPPG signals with two stages. In the first stage, we estimate the initial rPPG signals by contrastive learning from both the training data and their augmented counterparts. In the second stage, we use the estimated interference features to derive de-interfered rPPG features and encourage the rPPG signals to be distinct from the interference. In addition, we propose an effective descriptive rPPG feature learning by developing a strong 3D Learnable Descriptive Convolution (3DLDC) to capture the subtle chrominance changes for enhancing rPPG estimation. Extensive experiments conducted on five rPPG benchmark datasets demonstrate that the proposed DD-rPPGNet outperforms previous unsupervised rPPG estimation methods and achieves competitive performances with state-of-the-art supervised rPPG methods.
翻译:远程光电容积描记术(rPPG)旨在通过面部视频测量生理信号与心率(HR)。近期的无监督rPPG估计方法已展现出从面部区域估计rPPG信号而不依赖真实rPPG信号的潜力。然而,这些方法似乎忽视了rPPG信号中存在的干扰,导致性能仍不理想。本文提出一种新颖的去干扰描述性rPPG估计网络(DD-rPPGNet),通过消除rPPG特征中的干扰来学习真实的rPPG信号。首先,我们研究了干扰的局部时空相似性特征,并设计了一种新颖的无监督模型来估计干扰。接着,我们提出一种包含两个阶段的无监督去干扰方法来学习真实rPPG信号。在第一阶段,我们通过对比学习从训练数据及其增强样本中估计初始rPPG信号。在第二阶段,我们利用估计的干扰特征推导出去干扰的rPPG特征,并促使rPPG信号与干扰区分开来。此外,我们通过开发一种强大的三维可学习描述性卷积(3DLDC)来捕捉细微的色度变化,从而提出一种有效的描述性rPPG特征学习方法以增强rPPG估计。在五个rPPG基准数据集上进行的大量实验表明,所提出的DD-rPPGNet优于先前的无监督rPPG估计方法,并与最先进的监督式rPPG方法取得了具有竞争力的性能。