In recent years, the introduction of self-supervised contrastive learning (SSCL) has demonstrated remarkable improvements in representation learning across various domains, including natural language processing and computer vision. By leveraging the inherent benefits of self-supervision, SSCL enables the pre-training of representation models using vast amounts of unlabeled data. Despite these advances, there remains a significant gap in understanding the impact of different SSCL strategies on time series forecasting performance, as well as the specific benefits that SSCL can bring. This paper aims to address these gaps by conducting a comprehensive analysis of the effectiveness of various training variables, including different SSCL algorithms, learning strategies, model architectures, and their interplay. Additionally, to gain deeper insights into the improvements brought about by SSCL in the context of time-series forecasting, a qualitative analysis of the empirical receptive field is performed. Through our experiments, we demonstrate that the end-to-end training of a Transformer model using the Mean Squared Error (MSE) loss and SSCL emerges as the most effective approach in time series forecasting. Notably, the incorporation of the contrastive objective enables the model to prioritize more pertinent information for forecasting, such as scale and periodic relationships. These findings contribute to a better understanding of the benefits of SSCL in time series forecasting and provide valuable insights for future research in this area.

翻译：近年来，自监督对比学习（SSCL）的引入在自然语言处理和计算机视觉等多个领域的表征学习中展现出显著改进。通过利用自监督的固有优势，SSCL能够使用大量未标记数据对表征模型进行预训练。尽管取得了这些进展，但在理解不同SSCL策略对时间序列预测性能的影响，以及SSCL所能带来的具体益处方面，仍存在显著空白。本文旨在通过全面分析各种训练变量的有效性（包括不同的SSCL算法、学习策略、模型架构及其相互作用）来填补这些空白。此外，为更深入地理解SSCL在时间序列预测中带来的改进，我们对经验感受野进行了定性分析。通过实验，我们证明使用均方误差（MSE）损失函数和SSCL对Transformer模型进行端到端训练，是时间序列预测中最有效的方法。值得注意的是，引入对比目标能使模型优先关注更相关的预测信息，例如尺度关系和周期关系。这些发现有助于更深入地理解SSCL在时间序列预测中的优势，并为该领域的未来研究提供宝贵见解。