Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.

翻译：生物信号迁移学习近年来已成为提升小规模生物信号数据集下游任务预测性能的重要技术。近期研究表明，通过在大型数据集（如EEG）上使用自监督任务预训练神经网络模型，将自监督头替换为线性分类头，并在不同下游生物信号数据集（如EMG或ECG）上微调模型，能显著提升这些数据集的性能。本文提出一种用于低数据生物信号迁移学习的卷积-Transformer混合模型架构，引入基于频率的掩码自编码任务，采用更全面的评估框架，并系统评估（多模态）预训练在何种程度及何时能提升微调性能。我们还提出一种显著更高效的时序对齐方法，用于处理下游数据集与原始预训练数据集在时间长度和采样率上的差异。研究结果表明，混合模型中纯卷积部分能在某些低数据下游任务上达到最先进性能，而完整模型通常能进一步提升性能。对于基于Transformer的模型，我们发现预训练尤其能提升下游数据集性能，多模态预训练往往能进一步增强收益，且我们提出的基于频率的预训练方法在最低和最高数据量场景下平均表现最佳。