Spatio-Temporal forecasting is crucial in diverse fields, such as transportation, climate, and energy. Urban spatio-temporal data exhibits temporal mirage: similar short-window inputs have divergent future trends, and vice versa. Existing spatio-temporal graph neural networks (STGNNs) cannot effectively identify such mirages. We argue that the core reason lies in the short-window inputs that have incomplete period observation, heterogeneous global spatial correlation, and cross-period superposition causality. To bridge this gap, we develop a novel Multi- Period Pattern Pre-training (MP3), a plug-and-play pre-training plugin for distinguishing temporal mirages. MP3 presents two core innovations: (1) The multi-period pattern learning is designed to learn multi-period patterns from long time series. Specifically, multi-period temporal modeling leverages edge convolution to identify different multi-period patterns. Multi-period spatial modeling uses a bottleneck project and a global memory bank to capture heterogeneous global spatial relations efficiently. Cross-period pattern interaction employs a causality-enhanced Transformer to capture dependencies across different period patterns. (2) This plugin can seamlessly integrate into existing STGNN backbones to strengthen their forecasting performance. The experiment on five STGNN baselines across five real-world datasets (including a large-scale dataset CA) verify the effectiveness, superior scalability and strong adaptability of MP3, which brings consistent and robust performance improvements across all evaluated baselines. On average, MP3 reduces the MAE 4.7% and the RMSE 5.0%. The code can be available at https://github.com/YAN-outlook/MP3.
翻译:时空预测在交通、气候和能源等多个领域至关重要。城市时空数据呈现“时间幻象”:相似的短窗口输入可能产生截然不同的未来趋势,反之亦然。现有STGNN无法有效识别此类幻象。我们认为其核心原因在于短窗口输入存在周期观测不完整、全局空间关联异质以及跨周期叠加因果关系。为弥合这一差距,我们提出了一种新颖的多周期模式预训练(MP3)——一种用于区分时间幻象的即插即用式预训练插件。MP3包含两大核心创新:(1)多周期模式学习旨在从长时间序列中学习多周期模式。具体而言,多周期时序建模利用边卷积识别不同的多周期模式;多周期空间建模采用瓶颈投影和全局记忆库高效捕获异质全局空间关联;跨周期模式交互通过因果增强型Transformer捕获不同周期模式间的依赖关系。(2)该插件可无缝集成到现有STGNN骨干网络中,以增强其预测性能。在涵盖五个真实世界数据集(含大规模数据集CA)的五种STGNN基准模型上的实验验证了MP3的有效性、卓越可扩展性和强适应性,其能为所有评估基准带来一致且稳健的性能提升。平均而言,MP3使MAE降低4.7%,RMSE降低5.0%。代码可在https://github.com/YAN-outlook/MP3获取。