Recently, graph neural networks (GNNs) have shown its unprecedented success in many graph-related tasks. However, GNNs face the label scarcity issue as other neural networks do. Thus, recent efforts try to pre-train GNNs on a large-scale unlabeled graph and adapt the knowledge from the unlabeled graph to the target downstream task. The adaptation is generally achieved by fine-tuning the pre-trained GNNs with a limited number of labeled data. Despite the importance of fine-tuning, current GNNs pre-training works often ignore designing a good fine-tuning strategy to better leverage transferred knowledge and improve the performance on downstream tasks. Only few works start to investigate a better fine-tuning strategy for pre-trained GNNs. But their designs either have strong assumptions or overlook the data-aware issue for various downstream datasets. Therefore, we aim to design a better fine-tuning strategy for pre-trained GNNs to improve the model performance in this paper. Given a pre-trained GNN, we propose to search to fine-tune pre-trained graph neural networks for graph-level tasks (S2PGNN), which adaptively design a suitable fine-tuning framework for the given labeled data on the downstream task. To ensure the improvement brought by searching fine-tuning strategy, we carefully summarize a proper search space of fine-tuning framework that is suitable for GNNs. The empirical studies show that S2PGNN can be implemented on the top of 10 famous pre-trained GNNs and consistently improve their performance. Besides, S2PGNN achieves better performance than existing fine-tuning strategies within and outside the GNN area. Our code is publicly available at \url{https://anonymous.4open.science/r/code_icde2024-A9CB/}.

翻译：最近，图神经网络（GNN）在图相关任务中展现出前所未有的成功。然而，GNN与其他神经网络一样面临标签稀缺问题。因此，近期研究尝试在大规模无标签图上预训练GNN，并将无标签图的知识迁移至下游目标任务。这种迁移通常通过使用有限标注数据微调预训练GNN实现。尽管微调至关重要，当前GNN预训练工作普遍忽略设计良好的微调策略以更好地利用迁移知识并提升下游任务性能。仅有少数工作开始探索针对预训练GNN的更优微调策略，但其设计要么存在强假设，要么忽视不同下游数据集的数据感知问题。为此，本文旨在为预训练GNN设计更优微调策略以提升模型性能。针对给定预训练GNN，我们提出面向图级任务的预训练图神经网络搜索微调方法（S2PGNN），该方法能为下游任务中的给定标注数据自适应设计合适的微调框架。为确保搜索微调策略带来的性能提升，我们系统总结了适用于GNN的微调框架搜索空间。实验表明，S2PGNN可部署在10种知名预训练GNN之上，并持续提升其性能。此外，S2PGNN在GNN领域内外均优于现有微调策略。我们的代码已公开于\url{https://anonymous.4open.science/r/code_icde2024-A9CB/}。