Multivariate time series forecasting focuses on predicting future values based on historical context. State-of-the-art sequence-to-sequence models rely on neural attention between timesteps, which allows for temporal learning but fails to consider distinct spatial relationships between variables. In contrast, methods based on graph neural networks explicitly model variable relationships. However, these methods often rely on predefined graphs that cannot change over time and perform separate spatial and temporal updates without establishing direct connections between each variable at every timestep. Our work addresses these problems by translating multivariate forecasting into a "spatiotemporal sequence" formulation where each Transformer input token represents the value of a single variable at a given time. Long-Range Transformers can then learn interactions between space, time, and value information jointly along this extended sequence. Our method, which we call Spacetimeformer, achieves competitive results on benchmarks from traffic forecasting to electricity demand and weather prediction while learning spatiotemporal relationships purely from data.

翻译：多变量时间序列预测聚焦于基于历史上下文预测未来数值。最先进的序列到序列模型依赖时间步之间的神经注意力机制，虽能实现时序学习，但未能考虑变量间不同的空间关系。相比之下，基于图神经网络的方法显式建模变量关系，然而这些方法往往依赖无法随时间变化的预定义图结构，且分别进行空间与时间更新，未能建立每个变量在每个时间步的直接联系。本研究通过将多变量预测转化为"时空序列"表述形式解决上述问题——每个Transformer输入令牌代表特定时间单个变量的数值。长时间序列Transformer能够沿扩展序列联合学习空间、时间与数值信息的交互作用。我们提出的方法Spacetimeformer在从交通流预测到电力需求及天气预报等基准测试中均取得竞争性结果，同时完全从数据中学习时空关系。