The principal benefit of unsupervised graph representation learning is that a pre-trained model can be fine-tuned where data or labels are scarce. Existing approaches are domain specific, maintaining consistent node and edge attributes across the pre-training and target datasets. This precludes transfer to other domains. A model capable of positive transfer on arbitrary tasks and domains would represent the first foundation graph model. In this work we use adversarial contrastive learning to present FoToM, a graph pre-training method based on node and edge feature exclusion. We use FoToM to pre-train models over multiple graph domains, producing the first foundation graph models. We demonstrate positive transfer on evaluation datasets from multiple domains, including domains not present in pre-training data. On all datasets performance is at worst on-par and on 76% significantly better than a supervised baseline ($P \leq 0.01$), with an 8 to 40% reduction in error at 95% confidence. Contrary to other research, pre-training on a dataset with the target domain excluded leads us to better performance than pre-training on a dataset from only the target domain. The multi-domain model at worst, matches, and on 56% of tasks, significantly outperforms single-domain ($P \leq 0.01$). These results include when node labels are used in evaluation, where performance is consistently superior to single-domain or non-pre-trained models. Notably, FoToM benefits scenarios in both large or scarce data regimes for the target domains.

翻译：无监督图表示学习的主要优势在于，预训练模型可在数据或标签稀缺的场景下进行微调。现有方法具有领域特异性，要求在预训练数据集与目标数据集之间保持节点和边属性的一致性，这阻碍了模型向其他领域的迁移。能对任意任务和领域实现正向迁移的模型将构成首个基础图模型。本研究采用对抗性对比学习，提出基于节点与边特征排除的图预训练方法FoToM。我们利用FoToM在多个图领域上预训练模型，构建出首批基础图模型。在来自多个领域（含预训练数据未覆盖领域）的评估数据集上验证了正向迁移效果。所有数据集上的性能最差与有监督基线持平，其中76%的任务显著优于基线（$P \leq 0.01$），在95%置信度下误差降低8%至40%。与其他研究结论相反，排除目标领域的数据集上预训练反而比仅在目标领域数据集上预训练获得更优性能。多领域模型在最差情况下与单领域模型性能持平，且在56%的任务中显著优于单领域模型（$P \leq 0.01$）。这些结果还包括在评估中使用节点标签的场景，此时性能始终优于单领域或非预训练模型。值得注意的是，FoToM在目标领域的大数据或稀缺数据场景中均能获益。