Traffic forecasting has emerged as a crucial research area in the development of smart cities. Although various neural networks with intricate architectures have been developed to address this problem, they still face two key challenges: i) Recent advancements in network designs for modeling spatio-temporal correlations are starting to see diminishing returns in performance enhancements. ii) Additionally, most models do not account for the spatio-temporal heterogeneity inherent in traffic data, i.e., traffic distribution varies significantly across different regions and traffic flow patterns fluctuate across various time slots. To tackle these challenges, we introduce the Spatio-Temporal Graph Transformer (STGormer), which effectively integrates attribute and structure information inherent in traffic data for learning spatio-temporal correlations, and a mixture-of-experts module for capturing heterogeneity along spaital and temporal axes. Specifically, we design two straightforward yet effective spatial encoding methods based on the graph structure and integrate time position encoding into the vanilla transformer to capture spatio-temporal traffic patterns. Additionally, a mixture-of-experts enhanced feedforward neural network (FNN) module adaptively assigns suitable expert layers to distinct patterns via a spatio-temporal gating network, further improving overall prediction accuracy. Experiments on five real-world datasets demonstrate that STGormer achieves state-of-the-art performance.

翻译：交通流量预测已成为智慧城市发展中的关键研究领域。尽管已开发出多种具有复杂架构的神经网络来解决此问题，但它们仍面临两大挑战：i) 当前用于建模时空相关性的网络设计在性能提升方面开始呈现收益递减趋势；ii) 此外，大多数模型未能考虑交通数据固有的时空异质性，即交通分布在不同区域差异显著，且交通流模式在不同时段波动明显。为应对这些挑战，我们提出了时空图Transformer（STGormer），该模型能有效整合交通数据固有的属性与结构信息以学习时空相关性，并通过专家混合模块捕获时空维度的异质性。具体而言，我们基于图结构设计了两种简洁高效的空间编码方法，并将时间位置编码集成到标准Transformer中以捕获时空交通模式。此外，通过专家混合增强的前馈神经网络模块，借助时空门控网络将适配的专家层动态分配给不同模式，从而进一步提升整体预测精度。在五个真实数据集上的实验表明，STGormer实现了最先进的预测性能。