Graph Neural Networks (GNNs) have achieved great success in learning with graph-structured data. Privacy concerns have also been raised for the trained models which could expose the sensitive information of graphs including both node features and the structure information. In this paper, we aim to achieve node-level differential privacy (DP) for training GNNs so that a node and its edges are protected. Node DP is inherently difficult for GNNs because all direct and multi-hop neighbors participate in the calculation of gradients for each node via layer-wise message passing and there is no bound on how many direct and multi-hop neighbors a node can have, so existing DP methods will result in high privacy cost or poor utility due to high node sensitivity. We propose a \textbf{D}ecoupled GNN with Differentially \textbf{P}rivate \textbf{A}pproximate Personalized Page\textbf{R}ank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff. The key idea is to decouple the feature projection and message passing via a DP PageRank algorithm which learns the structure information and uses the top-$K$ neighbors determined by the PageRank for feature aggregation. By capturing the most important neighbors for each node and avoiding the layer-wise message passing, it bounds the node sensitivity and achieves improved privacy-utility tradeoff compared to layer-wise perturbation based methods. We theoretically analyze the node DP guarantee for the two processes combined together and empirically demonstrate better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.

翻译：图神经网络（GNN）在处理图结构数据的学习任务中取得了巨大成功。然而，训练后的模型可能暴露图的敏感信息（包括节点特征和结构信息），从而引发隐私问题。本文旨在实现训练GNN时的节点级差分隐私（DP），以保护节点及其边。节点DP对GNN而言具有内在困难性，因为所有直接邻居和多跳邻居通过逐层消息传递参与每个节点的梯度计算，且节点可能拥有的直接邻居和多跳邻居数量无上限，因此现有DP方法会因节点敏感度较高而导致高昂的隐私成本或低效用。我们提出一种基于差分隐私近似个性化PageRank的解耦GNN（DPAR），用于在增强隐私-效用权衡的条件下训练GNN。其核心思想是通过DP PageRank算法解耦特征投影与消息传递：该算法学习结构信息，并利用PageRank确定的前K个邻居进行特征聚合。通过捕获每个节点最重要的邻居并避免逐层消息传递，该方法限制了节点敏感度，相比基于逐层扰动的方案实现了更优的隐私-效用权衡。我们从理论上分析了两个合并过程的节点DP保证，并通过实验证明，在相同节点DP等级下，DPAR的效用优于当前最优方法。