In recent years, graph pre-training has gained significant attention, focusing on acquiring transferable knowledge from unlabeled graph data to improve downstream performance. Despite these recent endeavors, the problem of negative transfer remains a major concern when utilizing graph pre-trained models to downstream tasks. Previous studies made great efforts on the issue of what to pre-train and how to pre-train by designing a variety of graph pre-training and fine-tuning strategies. However, there are cases where even the most advanced "pre-train and fine-tune" paradigms fail to yield distinct benefits. This paper introduces a generic framework W2PGNN to answer the crucial question of when to pre-train (i.e., in what situations could we take advantage of graph pre-training) before performing effortful pre-training or fine-tuning. We start from a new perspective to explore the complex generative mechanisms from the pre-training data to downstream data. In particular, W2PGNN first fits the pre-training data into graphon bases, each element of graphon basis (i.e., a graphon) identifies a fundamental transferable pattern shared by a collection of pre-training graphs. All convex combinations of graphon bases give rise to a generator space, from which graphs generated form the solution space for those downstream data that can benefit from pre-training. In this manner, the feasibility of pre-training can be quantified as the generation probability of the downstream data from any generator in the generator space. W2PGNN offers three broad applications: providing the application scope of graph pre-trained models, quantifying the feasibility of pre-training, and assistance in selecting pre-training data to enhance downstream performance. We provide a theoretically sound solution for the first application and extensive empirical justifications for the latter two applications.

翻译：近年来，图预训练受到广泛关注，其核心目标是从无标注图数据中获取可迁移知识以提升下游任务性能。尽管已有诸多探索，但利用图预训练模型处理下游任务时，负迁移问题仍是主要挑战。现有研究通过设计多种图预训练与微调策略，重点解决了“预训练什么”与“如何预训练”的问题。然而，即便采用最先进的“预训练-微调”范式，在某些场景下仍无法获得显著收益。本文提出通用框架W2PGNN，旨在回答一个关键问题——“何时进行预训练”（即在何种情境下能真正受益于图预训练），从而避免无效的预训练或微调开销。我们从全新视角出发，探究预训练数据到下游数据的复杂生成机制。具体而言，W2PGNN首先将预训练数据拟合至图基元（graphon basis）中，每个图基元元素（即一个图基）表征一组预训练图共享的基础可迁移模式。所有图基基元的凸组合构成生成器空间，该空间生成的图形成下游数据（可受益于预训练）的解空间。通过此方式，预训练的可行性可量化为下游数据从生成器空间中任一生成器获得的生成概率。W2PGNN提供三大通用应用：界定图预训练模型的应用范围、量化预训练可行性、辅助选择预训练数据以提升下游性能。我们为首个应用提供了理论严谨的解决方案，并为后两个应用提供了充分的实验验证。