In natural language processing and vision, pretraining is utilized to learn effective representations. Unfortunately, the success of pretraining does not easily carry over to time series due to potential mismatch between sources and target. Actually, common belief is that multi-dataset pretraining does not work for time series! Au contraire, we introduce a new self-supervised contrastive pretraining approach to learn one encoding from many unlabeled and diverse time series datasets, so that the single learned representation can then be reused in several target domains for, say, classification. Specifically, we propose the XD-MixUp interpolation method and the Soft Interpolation Contextual Contrasting (SICC) loss. Empirically, this outperforms both supervised training and other self-supervised pretraining methods when finetuning on low-data regimes. This disproves the common belief: We can actually learn from multiple time series datasets, even from 75 at once.

翻译：在自然语言处理和视觉领域，预训练被用于学习有效的表示。然而，预训练的成功难以直接迁移至时间序列领域，因为源域和目标域之间可能存在不匹配。实际上，普遍观点认为多数据集预训练对时间序列无效！恰恰相反，我们提出了一种新的自监督对比预训练方法，从多个未标注且多样化的时间序列数据集中学习单一编码，使得该统一学习到的表示可重复用于多个目标域，例如分类任务。具体而言，我们提出了XD-MixUp插值方法和软插值上下文对比（SICC）损失。实验表明，在低数据场景下微调时，该方法优于监督训练及其他自监督预训练方法。这推翻了普遍认知：我们实际上可以从多个时间序列数据集中学习，甚至同时从75个数据集中学习。