Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. A corresponding resource that has been continuously updated can be found in the GitHub repository. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.

翻译：Transformer在自然语言处理和计算机视觉的众多任务中取得了优越的性能，这也激发了时间序列领域的浓厚兴趣。在Transformer的诸多优势中，捕获长程依赖关系和交互的能力对时间序列建模尤其具有吸引力，从而在各类时间序列应用中取得了令人兴奋的进展。本文系统性地回顾了用于时间序列建模的Transformer方案，重点强调了其优势与局限性。具体而言，我们从两个视角审视时间序列Transformer的发展：从网络结构视角，我们总结了为应对时间序列分析挑战而对Transformer进行的适配与修改；从应用视角，我们基于常见任务（包括预测、异常检测和分类）对时间序列Transformer进行了分类。在实验层面，我们通过鲁棒性分析、模型规模分析和季节-趋势分解分析来研究Transformer在时间序列中的表现。最后，我们讨论并提出了未来方向，以提供有用的研究指导。一个持续更新的配套资源可在GitHub仓库中找到。据我们所知，本文是首篇全面且系统性地总结Transformer在时间序列数据建模中最新进展的工作。我们希望这篇综述能够激发更多关于时间序列Transformer的研究兴趣。