Despite recent advances in Graph Neural Networks (GNNs), their training strategies remain largely under-explored. The conventional training strategy learns over all nodes in the original graph(s) equally, which can be sub-optimal as certain nodes are often more difficult to learn than others. Here we present TuneUp, a simple curriculum-based training strategy for improving the predictive performance of GNNs. TuneUp trains a GNN in two stages. In the first stage, TuneUp applies conventional training to obtain a strong base GNN. The base GNN tends to perform well on head nodes (nodes with large degrees) but less so on tail nodes (nodes with small degrees). Therefore, the second stage of TuneUp focuses on improving prediction on the difficult tail nodes by further training the base GNN on synthetically generated tail node data. We theoretically analyze TuneUp and show it provably improves generalization performance on tail nodes. TuneUp is simple to implement and applicable to a broad range of GNN architectures and prediction tasks. Extensive evaluation of TuneUp on five diverse GNN architectures, three types of prediction tasks, and both transductive and inductive settings shows that TuneUp significantly improves the performance of the base GNN on tail nodes, while often even improving the performance on head nodes. Altogether, TuneUp produces up to 57.6% and 92.2% relative predictive performance improvement in the transductive and the challenging inductive settings, respectively.

翻译：尽管图神经网络（GNN）近期取得了进展，但其训练策略仍鲜有深入研究。传统训练策略对原始图中所有节点一视同仁地学习，但某些节点往往比其他节点更难学习，这可能导致次优性能。本文提出TuneUp——一种基于课程学习的简单训练策略，用于提升GNN的预测性能。TuneUp分两阶段训练GNN：第一阶段采用常规训练获得强基GNN。该基GNN在头节点（大度数节点）上表现良好，但在尾节点（小度数节点）上效果欠佳。因此，TuneUp第二阶段通过进一步在合成生成的尾节点数据上训练基GNN，聚焦于提升困难尾节点的预测性能。我们从理论上分析了TuneUp，证明其可明确提升尾节点的泛化性能。TuneUp实现简单，适用于广泛的GNN架构与预测任务。在五种不同GNN架构、三类预测任务及直推式与归纳式设置下的广泛评估表明：TuneUp显著提升了基GNN在尾节点上的性能，同时通常还能改善头节点性能。总体而言，在直推式和具有挑战性的归纳式设置中，TuneUp分别带来了高达57.6%和92.2%的相对预测性能提升。