The principal benefit of unsupervised representation learning is that a pre-trained model can be fine-tuned where data or labels are scarce. Existing approaches for graph representation learning are domain specific, maintaining consistent node and edge features across the pre-training and target datasets. This has precluded transfer to multiple domains. In this work we present Topology Only Pre-Training, a graph pre-training method based on node and edge feature exclusion. Separating graph learning into two stages, topology and features, we use contrastive learning to pre-train models over multiple domains. These models show positive transfer on evaluation datasets from multiple domains, including domains not present in pre-training data. On 75% of experiments, ToP models perform significantly ($P \leq 0.01$) better than a supervised baseline. These results include when node and edge features are used in evaluation, where performance is significantly better on 85.7% of tasks compared to single-domain or non-pre-trained models. We further show that out-of-domain topologies can produce more useful pre-training than in-domain. We show better transfer from non-molecule pre-training, compared to molecule pre-training, on 79% of molecular benchmarks.

翻译：无监督表示学习的主要优势在于，预训练模型可以在数据或标签稀缺时进行微调。现有的图表示学习方法具有领域特异性，在预训练和目标数据集中保持一致的节点和边特征，这阻碍了模型向多领域的迁移。本文提出了一种基于排除节点和边特征的图预训练方法——仅拓扑预训练（Topology Only Pre-Training, ToP）。通过将图学习分为拓扑和特征两个阶段，我们利用对比学习在多个领域上预训练模型。这些模型在来自多个领域（包括预训练数据中未出现的领域）的评估数据集上展现出正向迁移效果。在75%的实验任务中，ToP模型的性能显著优于监督基线（P ≤ 0.01），且当评估中引入节点和边特征时，与单领域或非预训练模型相比，在85.7%的任务中性能显著更优。我们进一步证明，相较于域内拓扑，域外拓扑可产生更有价值的预训练效果；在79%的分子基准测试中，非分子预训练的迁移效果优于分子预训练。