Time-Series Mining (TSM) is an important research area since it shows great potential in practical applications. Deep learning models that rely on massive labeled data have been utilized for TSM successfully. However, constructing a large-scale well-labeled dataset is difficult due to data annotation costs. Recently, Pre-Trained Models have gradually attracted attention in the time series domain due to their remarkable performance in computer vision and natural language processing. In this survey, we provide a comprehensive review of Time-Series Pre-Trained Models (TS-PTMs), aiming to guide the understanding, applying, and studying TS-PTMs. Specifically, we first briefly introduce the typical deep learning models employed in TSM. Then, we give an overview of TS-PTMs according to the pre-training techniques. The main categories we explore include supervised, unsupervised, and self-supervised TS-PTMs. Further, extensive experiments are conducted to analyze the advantages and disadvantages of transfer learning strategies, Transformer-based models, and representative TS-PTMs. Finally, we point out some potential directions of TS-PTMs for future work.

翻译：时间序列挖掘是一个重要的研究领域，因为它在实际应用中展现出巨大潜力。依赖大量标注数据的深度学习模型已被成功应用于时间序列挖掘。然而，由于数据标注成本较高，构建大规模高质量标注数据集存在困难。近年来，预训练模型因其在计算机视觉和自然语言处理领域的卓越表现，逐渐在时间序列领域引起关注。本综述对时间序列预训练模型进行了全面回顾，旨在指导对时间序列预训练模型的理解、应用和研究。具体而言，我们首先简要介绍了时间序列挖掘中常用的典型深度学习模型。随后，根据预训练技术对时间序列预训练模型进行了概述。我们探讨的主要类别包括有监督、无监督和自监督时间序列预训练模型。此外，我们开展了大量实验，分析了迁移学习策略、基于Transformer的模型以及代表性时间序列预训练模型的优缺点。最后，我们指出了时间序列预训练模型未来研究的一些潜在方向。