Graphs have become an important modeling tool for Web applications, and graph neural networks (GNNs) have achieved great success in graph representation learning. However, their performance heavily relies on a large amount of supervision. Recently, ``pre-train, fine-tune'' has become the paradigm to address the issues of label dependency and poor generalization. However, the pre-training strategies vary for graphs with homophily and heterophily, and the objectives for various downstream tasks also differ. This leads to a gap between pretexts and downstream tasks, resulting in ``negative transfer'' and poor performance. Inspired by prompt learning in natural language processing, many studies turn to bridge the gap and fully leverage the pre-trained model. However, existing methods for graph prompting are tailored to homophily, neglecting inherent heterophily on graphs. Meanwhile, most of them rely on randomly initialized prompts, which negatively impact on the stability. Therefore, we propose Self-Prompt, a prompting framework for graphs based on the model and data itself. We first introduce asymmetric graph contrastive learning as pretext to address heterophily and align the objectives of pretext and downstream tasks. Then we reuse the component from pre-training as the self adapter and introduce self-prompts based on graph itself for task adaptation. Finally, we conduct extensive experiments on 11 benchmark datasets to demonstrate its superiority. We provide our codes at \url{https://github.com/gongchenghua/Self-Pro}.
翻译:图已成为网络应用中的重要建模工具,图神经网络(GNNs)在图表示学习领域取得了巨大成功。然而,其性能严重依赖于大量监督信息。近年来,“预训练-微调”范式已成为解决标签依赖性强与泛化能力不足的主要途径。但同配性与异配性图的预训练策略存在差异,且不同下游任务的目标函数亦不相同,这导致预训练任务与下游任务间存在间隙,引发“负迁移”并影响模型性能。受自然语言处理中提示学习的启发,许多研究致力于弥合该间隙以充分利用预训练模型。然而,现有图提示方法均针对同配性场景设计,忽视了图中固有的异配性特征;同时,这些方法大多依赖随机初始化的提示向量,会损害模型稳定性。为此,我们提出自提示框架——一种基于模型与数据自身的图提示方法。我们首先引入非对称图对比学习作为预训练任务以处理异配性,并实现预训练与下游任务的目标对齐;随后复用预训练组件作为自适应适配器,并基于图结构自身构建自提示向量以进行任务适配。最后,我们在11个基准数据集上进行了大量实验,验证了本方法的优越性。代码已开源:\url{https://github.com/gongchenghua/Self-Pro}。