Recently, graph neural networks (GNNs) have shown its unprecedented success in many graph-related tasks. However, GNNs face the label scarcity issue as other neural networks do. Thus, recent efforts try to pre-train GNNs on a large-scale unlabeled graph and adapt the knowledge from the unlabeled graph to the target downstream task. The adaptation is generally achieved by fine-tuning the pre-trained GNNs with a limited number of labeled data. Despite the importance of fine-tuning, current GNNs pre-training works often ignore designing a good fine-tuning strategy to better leverage transferred knowledge and improve the performance on downstream tasks. Only few works start to investigate a better fine-tuning strategy for pre-trained GNNs. But their designs either have strong assumptions or overlook the data-aware issue for various downstream datasets. Therefore, we aim to design a better fine-tuning strategy for pre-trained GNNs to improve the model performance in this paper. Given a pre-trained GNN, we propose to search to fine-tune pre-trained graph neural networks for graph-level tasks (S2PGNN), which adaptively design a suitable fine-tuning framework for the given labeled data on the downstream task. To ensure the improvement brought by searching fine-tuning strategy, we carefully summarize a proper search space of fine-tuning framework that is suitable for GNNs. The empirical studies show that S2PGNN can be implemented on the top of 10 famous pre-trained GNNs and consistently improve their performance. Besides, S2PGNN achieves better performance than existing fine-tuning strategies within and outside the GNN area. Our code is publicly available at \url{https://anonymous.4open.science/r/code_icde2024-A9CB/}.

翻译：最近，图神经网络（GNN）在众多图相关任务中展现出前所未有的成功。然而，与其他神经网络类似，GNN 面临标签稀缺的问题。为此，近期研究尝试在大规模无标签图上预训练 GNN，并将无标签图的知识迁移至目标下游任务。这种迁移通常通过使用有限数量的有标签数据对预训练 GNN 进行微调来实现。尽管微调至关重要，但当前的 GNN 预训练工作往往忽略设计良好的微调策略，以更好地利用迁移知识并提升下游任务性能。仅有少数研究开始探索预训练 GNN 的更优微调策略，但其设计要么存在强假设，要么忽视针对不同下游数据集的数据感知问题。因此，本文旨在为预训练 GNN 设计更优的微调策略以提升模型性能。给定预训练 GNN，我们提出面向图级任务的预训练图神经网络搜索微调方法（S2PGNN），该方法能为下游任务中的给定有标签数据自适应地设计合适的微调框架。为确保搜索微调策略带来的性能提升，我们精心总结了适用于 GNN 的微调框架搜索空间。实验表明，S2PGNN 可部署在 10 种知名预训练 GNN 之上，并持续提升其性能。此外，S2PGNN 在 GNN 领域内外的现有微调策略中均取得了更优性能。我们的代码已公开于 \url{https://anonymous.4open.science/r/code_icde2024-A9CB/}。