Temporal graph neural networks have shown promising results in learning inductive representations by automatically extracting temporal patterns. However, previous works often rely on complex memory modules or inefficient random walk methods to construct temporal representations. In addition, the existing dynamic graph encoders are non-trivial to adapt to self-supervised paradigms, which prevents them from utilizing unlabeled data. To address these limitations, we present an efficient yet effective attention-based encoder that leverages temporal edge encodings and window-based subgraph sampling to generate task-agnostic embeddings. Moreover, we propose a joint-embedding architecture using non-contrastive SSL to learn rich temporal embeddings without labels. Experimental results on 7 benchmark datasets indicate that on average, our model outperforms SoTA baselines on the future link prediction task by 4.23% for the transductive setting and 3.30% for the inductive setting while only requiring 5-10x less training/inference time. Additionally, we empirically validate the SSL pre-training significance under two probings commonly used in language and vision modalities. Lastly, different aspects of the proposed framework are investigated through experimental analysis and ablation studies.

翻译：时序图神经网络通过自动提取时间模式，在学习归纳式表示方面展现出令人瞩目的成果。然而，现有方法往往依赖复杂的记忆模块或低效的随机游走方法来构建时序表示。此外，现有动态图编码器难以适配自监督范式，从而限制了其对未标注数据的利用。为解决上述局限，我们提出了一种高效且有效的基于注意力机制的编码器，该编码器利用时序边编码与基于窗口的子图采样来生成与任务无关的嵌入表示。同时，我们提出了一种基于非对比式自监督学习的联合嵌入架构，无需标注即可学习丰富的时序嵌入。在7个基准数据集上的实验结果表明，我们的模型在直推式设置下平均未来链路预测性能超越现有最优基线4.23%，在归纳式设置下超越3.30%，同时训练/推理时间仅需其1/5至1/10。此外，我们通过语言与视觉模态常用的两种探测方法，实证验证了自监督预训练的重要性。最后，我们通过实验分析与消融研究，对提出框架的多个方面进行了深入探究。