Relational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data by representing it as a heterogeneous temporal graph. However, commonly used Graph Neural Network models suffer from fundamental limitations in capturing complex structural patterns and long-range dependencies that are inherent in relational data. While Graph Transformers have emerged as powerful alternatives to GNNs on general graphs, applying them to relational entity graphs presents unique challenges: (i) Traditional positional encodings fail to generalize to massive, heterogeneous graphs; (ii) existing architectures cannot model the temporal dynamics and schema constraints of relational data; (iii) existing tokenization schemes lose critical structural information. Here we introduce the Relational Graph Transformer (RelGT), the first graph transformer architecture designed specifically for relational tables. RelGT employs a novel multi-element tokenization strategy that decomposes each node into five components (features, type, hop distance, time, and local structure), enabling efficient encoding of heterogeneity, temporality, and topology without expensive precomputation. Our architecture combines local attention over sampled subgraphs with global attention to learnable centroids, incorporating both local and database-wide representations. Across 21 tasks from the RelBench benchmark, RelGT consistently matches or outperforms GNN baselines by up to 18%, establishing Graph Transformers as a powerful architecture for Relational Deep Learning.

翻译：关系深度学习（RDL）是一种通过将多表关系数据表示为异质时序图来构建最先进预测模型的有前景的方法。然而，常用的图神经网络模型在捕捉关系数据固有的复杂结构模式和长程依赖方面存在根本性局限。尽管图Transformer已作为通用图上GNN的强大替代方案出现，但将其应用于关系实体图仍面临独特挑战：（i）传统的位置编码难以泛化至大规模异质图；（ii）现有架构无法建模关系数据的时序动态与模式约束；（iii）现有标记化方案会丢失关键结构信息。本文提出关系图Transformer（RelGT），这是首个专为关系表设计的图Transformer架构。RelGT采用新颖的多元素标记化策略，将每个节点分解为五个组件（特征、类型、跳数距离、时间与局部结构），从而实现对异质性、时序性与拓扑结构的高效编码，且无需昂贵的预计算。该架构结合了采样子图上的局部注意力与可学习质心上的全局注意力，同时融合了局部与数据库级别的表征。在RelBench基准测试的21项任务中，RelGT始终匹配或超越GNN基线模型，最高提升达18%，确立了图Transformer作为关系深度学习的强大架构。