Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.

翻译：生物信号迁移学习近年来已成为提升小规模生物信号数据集下游任务预测性能的重要技术。近期研究表明，通过在大型数据集（如脑电图）上使用自监督任务预训练神经网络模型，将自监督头替换为线性分类头，并在不同下游生物信号数据集（如肌电图或心电图）上微调模型，可显著提升这些数据集的性能。本文提出一种新型卷积-Transformer混合模型架构，采用掩码自编码技术处理低数据生物信号迁移学习；引入基于频率的掩码自编码任务，构建更全面的评估框架，系统评估（多模态）预训练在何种程度及何时能提升微调性能。同时，我们提出一种性能显著提升的方法，用于将具有不同时间长度和采样率的下游数据集与原始预训练数据集对齐。研究结果表明：混合模型中纯卷积部分可在某些低数据下游任务中达到最先进性能，而完整模型通常能进一步提升性能。对于基于Transformer的模型，我们发现预训练尤其能提升下游数据集性能，多模态预训练往往能进一步扩大增益，且我们提出的基于频率的预训练方法在最低和最高数据量场景下平均表现最优。