The principal benefit of unsupervised representation learning is that a pre-trained model can be fine-tuned where data or labels are scarce. Existing approaches for graph representation learning are domain specific, maintaining consistent node and edge features across the pre-training and target datasets. This has precluded transfer to multiple domains. We present Topology Only Pre-Training (ToP), a graph pre-training method based on node and edge feature exclusion. We show positive transfer on evaluation datasets from multiple domains, including domains not present in pre-training data, running directly contrary to assumptions made in contemporary works. On 75% of experiments, ToP models perform significantly $p \leq 0.01$ better than a supervised baseline. Performance is significantly positive on 85.7% of tasks when node and edge features are used in fine-tuning. We further show that out-of-domain topologies can produce more useful pre-training than in-domain. Under ToP we show better transfer from non-molecule pre-training, compared to molecule pre-training, on 79% of molecular benchmarks. Against the limited set of other generalist graph models ToP performs strongly, including against models with many orders of magnitude larger. These findings show that ToP opens broad areas of research in both transfer learning on scarcely populated graph domains and in graph foundation models.

翻译：无监督表示学习的主要优势在于，预训练模型可在数据或标签稀缺的场景中进行微调。现有的图表示学习方法均局限于特定领域，要求预训练数据集与目标数据集保持一致的节点与边特征，这阻碍了跨多领域迁移的实现。本文提出仅拓扑预训练方法，这是一种基于节点与边特征排除的图预训练方法。实验证明，该方法在多个领域的评估数据集上均实现正向迁移，甚至包含预训练数据中未出现的领域，这与当前研究的基本假设直接相悖。在75%的实验中，ToP模型的表现显著优于监督基线（$p \leq 0.01$）。当微调阶段引入节点与边特征时，85.7%的任务呈现显著正向效果。研究进一步表明，领域外拓扑结构可能比领域内拓扑产生更有价值的预训练效果。在分子基准测试中，79%的实验显示非分子预训练比分子预训练具有更优的迁移性能。相较于其他有限的通用图模型（包括参数量高出数个数量级的模型），ToP均表现出强劲竞争力。这些发现证明，ToP为稀疏图领域的迁移学习及图基础模型研究开辟了广阔前景。