Transformer-based models for long sequence time series forecasting (LSTF) problems have gained significant attention due to their exceptional forecasting precision. As the cornerstone of these models, the self-attention mechanism poses a challenge to efficient training and inference due to its quadratic time complexity. In this article, we propose a novel architectural design for Transformer-based models in LSTF, leveraging a substitution framework that incorporates Surrogate Attention Blocks and Surrogate FFN Blocks. The framework aims to boost any well-designed model's efficiency without sacrificing its accuracy. We further establish the equivalence of the Surrogate Attention Block to the self-attention mechanism in terms of both expressiveness and trainability. Through extensive experiments encompassing nine Transformer-based models across five time series tasks, we observe an average performance improvement of 9.45% while achieving a significant reduction in model size by 46%

翻译：基于Transformer的长序列时间序列预测模型因其卓越的预测精度而受到广泛关注。作为这些模型的核心，自注意力机制因其二次时间复杂度对高效训练与推理构成了挑战。本文针对长序列时间序列预测中的Transformer类模型，提出了一种新颖的架构设计，通过引入包含替代注意力模块与替代前馈网络模块的替换框架，旨在提升任何设计优良模型的效率而不牺牲其精度。我们进一步从表达能力与可训练性两个维度证明了替代注意力模块与自注意力机制的等价性。通过在五个时间序列任务中对九种基于Transformer的模型进行广泛实验，我们观察到模型平均性能提升了9.45%，同时模型规模显著减少了46%。