Transformers have achieved remarkable performance in multivariate time series(MTS) forecasting due to their capability to capture long-term dependencies. However, the canonical attention mechanism has two key limitations: (1) its quadratic time complexity limits the sequence length, and (2) it generates future values from the entire historical sequence. To address this, we propose a Dozer Attention mechanism consisting of three sparse components: (1) Local, each query exclusively attends to keys within a localized window of neighboring time steps. (2) Stride, enables each query to attend to keys at predefined intervals. (3) Vary, allows queries to selectively attend to keys from a subset of the historical sequence. Notably, the size of this subset dynamically expands as forecasting horizons extend. Those three components are designed to capture essential attributes of MTS data, including locality, seasonality, and global temporal dependencies. Additionally, we present the Dozerformer Framework, incorporating the Dozer Attention mechanism for the MTS forecasting task. We evaluated the proposed Dozerformer framework with recent state-of-the-art methods on nine benchmark datasets and confirmed its superior performance. The experimental results indicate that excluding a subset of historical time steps from the time series forecasting process does not compromise accuracy while significantly improving efficiency. Code is available at https://github.com/GRYGY1215/Dozerformer.

翻译：Transformer凭借其捕捉长期依赖关系的能力，在多元时间序列预测中取得了显著性能。然而，标准注意力机制存在两个关键局限：(1) 其二次时间复杂度限制了序列长度；(2) 它基于整个历史序列生成未来值。为此，我们提出一种由三个稀疏组件构成的Dozer注意力机制：(1) 局部组件：每个查询仅关注相邻时间步局部窗口内的键。(2) 步幅组件：使每个查询能够关注预定义间隔的键。(3) 可变组件：允许查询有选择地关注历史序列子集中的键。值得注意的是，该子集的大小会随着预测时间范围的延伸而动态扩展。这三个组件旨在捕捉多元时间序列数据的关键特性，包括局部性、季节性和全局时间依赖性。此外，我们提出了集成Dozer注意力机制的Dozerformer框架，用于多元时间序列预测任务。我们在九个基准数据集上使用近期最先进方法评估了所提出的Dozerformer框架，并验证了其优越性能。实验结果表明，在时间序列预测过程中排除部分历史时间步不仅不会降低精度，还能显著提升效率。代码发布于https://github.com/GRYGY1215/Dozerformer。