Multivariate Time Series (MTS) forecasting involves modeling temporal dependencies within historical records. Transformers have demonstrated remarkable performance in MTS forecasting due to their capability to capture long-term dependencies. However, prior work has been confined to modeling temporal dependencies at either a fixed scale or multiple scales that exponentially increase (most with base 2). This limitation hinders their effectiveness in capturing diverse seasonalities, such as hourly and daily patterns. In this paper, we introduce a dimension invariant embedding technique that captures short-term temporal dependencies and projects MTS data into a higher-dimensional space, while preserving the dimensions of time steps and variables in MTS data. Furthermore, we present a novel Multi-scale Transformer Pyramid Network (MTPNet), specifically designed to effectively capture temporal dependencies at multiple unconstrained scales. The predictions are inferred from multi-scale latent representations obtained from transformers at various scales. Extensive experiments on nine benchmark datasets demonstrate that the proposed MTPNet outperforms recent state-of-the-art methods.

翻译：多变量时间序列（MTS）预测涉及对历史记录中时间依赖关系的建模。Transformer凭借其捕捉长期依赖关系的能力，在MTS预测中展现了卓越性能。然而，现有研究仅限于在固定尺度或呈指数增长（多以2为底）的多尺度下对时间依赖关系进行建模。这种局限性阻碍了其有效捕捉多样化季节模式（如小时级和日级模式）的能力。本文提出了一种维度不变的嵌入技术，该技术能够捕捉短期时间依赖关系，并将MTS数据投影至高维空间，同时保留MTS数据中时间步与变量的维度。此外，我们提出了一种新颖的多尺度Transformer金字塔网络（MTPNet），专门设计用于在不受约束的多尺度下有效捕捉时间依赖关系。预测结果通过从不同尺度的Transformer中获取的多尺度潜在表示推断得出。在九个基准数据集上进行的大量实验表明，所提出的MTPNet优于近期最先进方法。