Over the past decade, Time Series Classification (TSC) has gained an increasing attention. While various methods were explored, deep learning - particularly through Convolutional Neural Networks (CNNs)-stands out as an effective approach. However, due to the limited availability of training data, defining a foundation model for TSC that overcomes the overfitting problem is still a challenging task. The UCR archive, encompassing a wide spectrum of datasets ranging from motion recognition to ECG-based heart disease detection, serves as a prime example for exploring this issue in diverse TSC scenarios. In this paper, we address the overfitting challenge by introducing pre-trained domain foundation models. A key aspect of our methodology is a novel pretext task that spans multiple datasets. This task is designed to identify the originating dataset of each time series sample, with the goal of creating flexible convolution filters that can be applied across different datasets. The research process consists of two phases: a pre-training phase where the model acquires general features through the pretext task, and a subsequent fine-tuning phase for specific dataset classifications. Our extensive experiments on the UCR archive demonstrate that this pre-training strategy significantly outperforms the conventional training approach without pre-training. This strategy effectively reduces overfitting in small datasets and provides an efficient route for adapting these models to new datasets, thus advancing the capabilities of deep learning in TSC.

翻译：过去十年间，时间序列分类（TSC）逐渐引起了广泛关注。尽管已有多种研究方法被探索，但深度学习——特别是通过卷积神经网络（CNN）——已成为一种有效途径。然而，由于训练数据有限，如何构建能克服过拟合问题的TSC基础模型仍是一项挑战。涵盖从运动识别到基于心电图的心脏病检测等广泛数据集的UCR档案，为在不同TSC场景中研究该问题提供了典型范例。本文通过引入预训练的领域基础模型来解决过拟合挑战。我们方法论的关键要素是一种跨多个数据集的新型预文本任务。该任务旨在识别每个时间序列样本的原始数据集，目标是创建可跨不同数据集应用的灵活卷积滤波器。研究过程分为两个阶段：预训练阶段（模型通过预文本任务获取通用特征）和后续特定数据集分类的微调阶段。我们在UCR档案上的大量实验表明，这种预训练策略显著优于无预训练的传统训练方法。该策略有效减少了小数据集中的过拟合现象，并为将这些模型适配到新数据集提供了高效路径，从而提升了深度学习在TSC领域的能力。