Masked time series modeling has recently gained much attention as a self-supervised representation learning strategy for time series. Inspired by masked image modeling in computer vision, recent works first patchify and partially mask out time series, and then train Transformers to capture the dependencies between patches by predicting masked patches from unmasked patches. However, we argue that capturing such patch dependencies might not be an optimal strategy for time series representation learning; rather, learning to embed patches independently results in better time series representations. Specifically, we propose to use 1) the simple patch reconstruction task, which autoencode each patch without looking at other patches, and 2) the simple patch-wise MLP that embeds each patch independently. In addition, we introduce complementary contrastive learning to hierarchically capture adjacent time series information efficiently. Our proposed method improves time series forecasting and classification performance compared to state-of-the-art Transformer-based models, while it is more efficient in terms of the number of parameters and training/inference time. Code is available at this repository: https://github.com/seunghan96/pits.

翻译：掩码时间序列建模近来作为一种自监督表示学习策略在时间序列领域备受关注。受计算机视觉中掩码图像建模的启发，近期工作首先对时间序列进行片段化处理并部分掩蔽，随后训练Transformer模型通过从非掩蔽片段预测掩蔽片段来捕获片段间的依赖关系。然而，我们认为捕获此类片段依赖关系可能并非时间序列表示学习的最佳策略；相反，学习独立嵌入每个片段能产生更优的时间序列表示。具体而言，我们提出采用：1）简单的片段重建任务，即在不参考其他片段的情况下对每个片段进行自编码；2）简单的逐片段MLP，独立嵌入每个片段。此外，我们引入互补对比学习以高效层次化捕获相邻时间序列信息。与最先进的基于Transformer的模型相比，我们提出的方法在时间序列预测与分类性能上有所提升，同时在参数数量及训练/推理时间方面更具效率。代码已发布于以下仓库：https://github.com/seunghan96/pits。