Machine learning models for camera-based physiological measurement can have weak generalization due to a lack of representative training data. Body motion is one of the most significant sources of noise when attempting to recover the subtle cardiac pulse from a video. We explore motion transfer as a form of data augmentation to introduce motion variation while preserving physiological changes of interest. We adapt a neural video synthesis approach to augment videos for the task of remote photoplethysmography (rPPG) and study the effects of motion augmentation with respect to 1) the magnitude and 2) the type of motion. After training on motion-augmented versions of publicly available datasets, we demonstrate a 47% improvement over existing inter-dataset results using various state-of-the-art methods on the PURE dataset. We also present inter-dataset results on five benchmark datasets to show improvements of up to 79% using TS-CAN, a neural rPPG estimation method. Our findings illustrate the usefulness of motion transfer as a data augmentation technique for improving the generalization of models for camera-based physiological sensing. We release our code for using motion transfer as a data augmentation technique on three publicly available datasets, UBFC-rPPG, PURE, and SCAMPS, and models pre-trained on motion-augmented data here: https://motion-matters.github.io/

翻译：基于相机的生理测量机器学习模型因缺乏代表性训练数据而泛化能力不足。在尝试从视频中恢复微弱心脏脉搏信号时，身体运动是最主要的噪声源之一。我们探索将运动迁移作为一种数据增强手段，在保留目标生理变化的同时引入运动多样性。通过调整神经视频合成方法，我们为远程光电容积描记法(rPPG)任务增强视频数据，并系统研究了运动增强中1)运动幅度与2)运动类型的影响。在公开数据集上使用运动增强版本进行训练后，我们基于PURE数据集采用多种先进方法，将跨数据集性能提升47%。同时，我们在五个基准数据集上展示了跨数据集结果，其中使用神经rPPG估计方法TS-CAN获得了高达79%的性能提升。研究结果表明，运动迁移作为数据增强技术能有效提升基于相机的生理感知模型泛化能力。我们在https://motion-matters.github.io/ 开源了三项公开数据集（UBFC-rPPG、PURE和SCAMPS）的运动迁移数据增强代码及基于运动增强数据的预训练模型。