Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.

翻译：Transformer在自然语言处理和计算机视觉的众多任务中取得了卓越性能，这也在时间序列领域引发了极大兴趣。在Transformer的诸多优势中，捕获长程依赖与交互的能力对时间序列建模尤为吸引人，从而推动了各类时间序列应用取得令人振奋的进展。本文通过强调其优势与局限性，系统性地回顾了用于时间序列建模的Transformer方案。具体而言，我们从两个视角审视时间序列Transformer的发展：从网络结构视角，我们总结了为应对时间序列分析挑战而对Transformer进行的适配与修改；从应用视角，我们根据常见任务（包括预测、异常检测和分类）对时间序列Transformer进行了分类。在实证层面，我们通过鲁棒性分析、模型规模分析以及季节趋势分解分析，研究了Transformer在时间序列中的表现。最后，我们讨论并提出了未来方向，以提供有益的研究指导。据我们所知，本文是首篇全面且系统地总结Transformer在时间序列数据建模领域最新进展的工作。我们希望本综述能进一步激发时间序列Transformer的研究兴趣。