Facial remote photoplethysmography (rPPG) methods estimate physiological signals by modeling subtle color changes on the 3D facial surface over time. However, existing methods fail to explicitly align their receptive fields with the 3D facial surface-the spatial support of the rPPG signal. To address this, we propose the Facial Spatiotemporal Graph (STGraph), a novel representation that encodes facial color and structure using 3D facial mesh sequences-enabling surface-aligned spatiotemporal processing. We introduce MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals. Across four benchmark datasets, MeshPhys achieves state-of-the-art or competitive performance in both intra- and cross-dataset settings. Ablation studies show that constraining the model's receptive field to the facial surface acts as a strong structural prior, and that surface-aligned, 3D-aware node features are critical for robustly encoding facial surface color. Together, the STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG, enabling robust, interpretable, and generalizable estimation. Code is available at https://samcantrill.github.io/facial-stgraph-rppg/ .
翻译:面部远程光电容积描记法(rPPG)通过建模三维面部表面随时间变化的细微颜色变化来估计生理信号。然而,现有方法未能将其感受野与rPPG信号的空间支撑——三维面部表面——进行显式对齐。为解决此问题,我们提出了面部时空图(STGraph),这是一种利用三维面部网格序列编码面部颜色与结构的新型表示方法,实现了表面对齐的时空处理。我们引入了MeshPhys,一种轻量级的时空图卷积网络,其在STGraph上运行以估计生理信号。在四个基准数据集上,MeshPhys在数据集内和跨数据集设置中均达到了最先进或具有竞争力的性能。消融研究表明,将模型的感受野约束于面部表面可作为一种强结构先验,并且表面对齐、具备三维感知的节点特征对于稳健编码面部表面颜色至关重要。综上所述,STGraph与MeshPhys共同构成了一种新颖、原理驱动的面部rPPG建模范式,实现了稳健、可解释且可泛化的估计。代码可在 https://samcantrill.github.io/facial-stgraph-rppg/ 获取。