Machine learning models for camera-based physiological measurement can have weak generalization due to a lack of representative training data. Body motion is one of the most significant sources of noise when attempting to recover the subtle cardiac pulse from a video. We explore motion transfer as a form of data augmentation to introduce motion variation while preserving physiological changes of interest. We adapt a neural video synthesis approach to augment videos for the task of remote photoplethysmography (rPPG) and study the effects of motion augmentation with respect to 1) the magnitude and 2) the type of motion. After training on motion-augmented versions of publicly available datasets, we demonstrate a 47% improvement over existing inter-dataset results using various state-of-the-art methods on the PURE dataset. We also present inter-dataset results on five benchmark datasets to show improvements of up to 79% using TS-CAN, a neural rPPG estimation method. Our findings illustrate the usefulness of motion transfer as a data augmentation technique for improving the generalization of models for camera-based physiological sensing. We release our code for using motion transfer as a data augmentation technique on three publicly available datasets, UBFC-rPPG, PURE, and SCAMPS, and models pre-trained on motion-augmented data here: https://motion-matters.github.io/

翻译：基于摄像头的生理测量机器学习模型因缺乏代表性训练数据而泛化能力较弱。从视频中恢复微小的心跳脉搏信号时，身体运动是最主要的噪声来源之一。我们探索将运动迁移作为一种数据增强方法，在保留相关生理变化的同时引入运动多样性。通过改进神经视频合成方法，我们为远程光电容积描记（rPPG）任务增强视频数据，并从1）运动幅度和2）运动类型两方面研究运动增强的效果。在公开数据集的运动增强版本上训练后，我们在PURE数据集上采用多种最先进方法实现了比现有跨数据集结果提升47%的性能。同时，我们在五个基准数据集上展示了跨数据集结果，使用TS-CAN神经rPPG估计方法时性能提升最高达79%。研究结果表明，运动迁移作为数据增强技术能有效改善摄像机生理测量模型的泛化能力。我们在https://motion-matters.github.io/ 发布了三项公开数据集（UBFC-rPPG、PURE和SCAMPS）的运动迁移增强代码，以及基于运动增强数据预训练的模型。