Pre-trained graph models (PGMs) aim to capture transferable inherent structural properties and apply them to different downstream tasks. Similar to pre-trained language models, PGMs also inherit biases from human society, resulting in discriminatory behavior in downstream applications. The debiasing process of existing fair methods is generally coupled with parameter optimization of GNNs. However, different downstream tasks may be associated with different sensitive attributes in reality, directly employing existing methods to improve the fairness of PGMs is inflexible and inefficient. Moreover, most of them lack a theoretical guarantee, i.e., provable lower bounds on the fairness of model predictions, which directly provides assurance in a practical scenario. To overcome these limitations, we propose a novel adapter-tuning framework that endows pre-trained graph models with provable fairness (called GraphPAR). GraphPAR freezes the parameters of PGMs and trains a parameter-efficient adapter to flexibly improve the fairness of PGMs in downstream tasks. Specifically, we design a sensitive semantic augmenter on node representations, to extend the node representations with different sensitive attribute semantics for each node. The extended representations will be used to further train an adapter, to prevent the propagation of sensitive attribute semantics from PGMs to task predictions. Furthermore, with GraphPAR, we quantify whether the fairness of each node is provable, i.e., predictions are always fair within a certain range of sensitive attribute semantics. Experimental evaluations on real-world datasets demonstrate that GraphPAR achieves state-of-the-art prediction performance and fairness on node classification task. Furthermore, based on our GraphPAR, around 90\% nodes have provable fairness.

翻译：预训练图模型（PGMs）旨在捕捉可迁移的固有结构属性，并将其应用于不同的下游任务。与预训练语言模型类似，PGMs 也会继承人类社会的偏见，导致下游应用中出现歧视性行为。现有公平方法的去偏过程通常与图神经网络（GNNs）的参数优化耦合。然而，实际中不同的下游任务可能关联不同的敏感属性，直接采用现有方法提升 PGMs 的公平性既不灵活也低效。此外，大多数方法缺乏理论保证，即模型预测公平性的可证明下界，而这在实际场景中能直接提供保障。为克服这些局限，我们提出一种新颖的适配器微调框架，赋予预训练图模型可证明的公平性（称为 GraphPAR）。GraphPAR 冻结 PGMs 的参数，并训练一个参数高效的适配器，以灵活提升 PGMs 在下游任务中的公平性。具体而言，我们设计了一个基于节点表示的敏感语义增强器，为每个节点扩展具有不同敏感属性语义的节点表示。扩展后的表示将用于进一步训练适配器，以防止 PGMs 中的敏感属性语义传播至任务预测。此外，利用 GraphPAR，我们可量化每个节点的公平性是否可证明，即在一定敏感属性语义范围内，预测永远是公平的。在真实数据集上的实验评估表明，GraphPAR 在节点分类任务上实现了最先进的预测性能和公平性。同时，基于我们的 GraphPAR，约 90% 的节点具有可证明的公平性。