Pretrained Graph Neural Networks have been widely adopted for various molecular property prediction tasks. Despite their ability to encode structural and relational features of molecules, traditional fine-tuning of such pretrained GNNs on the target task can lead to poor generalization. To address this, we explore the adaptation of pretrained GNNs to the target task by jointly training them with multiple auxiliary tasks. This could enable the GNNs to learn both general and task-specific features, which may benefit the target task. However, a major challenge is to determine the relatedness of auxiliary tasks with the target task. To address this, we investigate multiple strategies to measure the relevance of auxiliary tasks and integrate such tasks by adaptively combining task gradients or by learning task weights via bi-level optimization. Additionally, we propose a novel gradient surgery-based approach, Rotation of Conflicting Gradients ($\mathtt{RCGrad}$), that learns to align conflicting auxiliary task gradients through rotation. Our experiments with state-of-the-art pretrained GNNs demonstrate the efficacy of our proposed methods, with improvements of up to 7.7% over fine-tuning. This suggests that incorporating auxiliary tasks along with target task fine-tuning can be an effective way to improve the generalizability of pretrained GNNs for molecular property prediction.

翻译：预训练图神经网络已被广泛应用于各种分子性质预测任务。尽管它们能够编码分子的结构和关系特征，但在目标任务上对这类预训练GNN进行传统微调可能会导致泛化能力不足。为解决这一问题，我们探索通过联合训练多个辅助任务来使预训练GNN适应目标任务。这种方法能使GNN同时学习通用特征和任务特定特征，从而有利于目标任务。然而，主要挑战在于确定辅助任务与目标任务的关联性。为此，我们研究了多种策略来度量辅助任务的相关性，并通过自适应组合任务梯度或利用双层优化学习任务权重来整合这些任务。此外，我们提出了一种基于梯度手术的新方法——冲突梯度旋转（$\mathtt{RCGrad}$），该方法通过学习旋转操作来对齐相互冲突的辅助任务梯度。我们在最先进的预训练GNN上进行的实验证明了所提方法的有效性，相比微调方法性能提升高达7.7%。这表明在目标任务微调的同时融入辅助任务，是提升预训练GNN在分子性质预测中泛化能力的一种有效途径。