In recent years, the introduction of self-supervised contrastive learning (SSCL) has demonstrated remarkable improvements in representation learning across various domains, including natural language processing and computer vision. By leveraging the inherent benefits of self-supervision, SSCL enables the pre-training of representation models using vast amounts of unlabeled data. Despite these advances, there remains a significant gap in understanding the impact of different SSCL strategies on time series forecasting performance, as well as the specific benefits that SSCL can bring. This paper aims to address these gaps by conducting a comprehensive analysis of the effectiveness of various training variables, including different SSCL algorithms, learning strategies, model architectures, and their interplay. Additionally, to gain deeper insights into the improvements brought about by SSCL in the context of time-series forecasting, a qualitative analysis of the empirical receptive field is performed. Through our experiments, we demonstrate that the end-to-end training of a Transformer model using the Mean Squared Error (MSE) loss and SSCL emerges as the most effective approach in time series forecasting. Notably, the incorporation of the contrastive objective enables the model to prioritize more pertinent information for forecasting, such as scale and periodic relationships. These findings contribute to a better understanding of the benefits of SSCL in time series forecasting and provide valuable insights for future research in this area. Our codes are available at https://github.com/chiyuzhang94/contrastive_learning_time-series_e2e.

翻译：近年来，自监督对比学习（SSCL）的引入在自然语言处理、计算机视觉等多个领域的表征学习上取得了显著进展。通过利用自监督的固有优势，SSCL能够使用大量无标签数据预训练表征模型。然而，不同SSCL策略对时间序列预测性能的影响，以及SSCL能够带来的具体益处，目前仍存在显著的理解空白。本文旨在通过全面分析各种训练变量（包括不同SSCL算法、学习策略、模型架构及其相互作用）的有效性来填补这些空白。此外，为了更深入地洞悉SSCL在时间序列预测中带来的改进，本文对经验感受野进行了定性分析。通过实验，我们证明使用均方误差（MSE）损失和SSCL进行端到端训练的Transformer模型，是时间序列预测中最有效的方法。值得注意的是，对比目标的引入使模型能够优先关注对预测更相关的信息，例如尺度和周期关系。这些发现有助于更深入地理解SSCL在时间序列预测中的优势，并为该领域的未来研究提供了宝贵见解。我们的代码可在 https://github.com/chiyuzhang94/contrastive_learning_time-series_e2e 获取。