Graph Neural Networks (GNNs) have achieved great success in learning with graph-structured data. Privacy concerns have also been raised for the trained models which could expose the sensitive information of graphs including both node features and the structure information. In this paper, we aim to achieve node-level differential privacy (DP) for training GNNs so that a node and its edges are protected. Node DP is inherently difficult for GNNs because all direct and multi-hop neighbors participate in the calculation of gradients for each node via layer-wise message passing and there is no bound on how many direct and multi-hop neighbors a node can have, so existing DP methods will result in high privacy cost or poor utility due to high node sensitivity. We propose a Decoupled GNN with Differentially Private Approximate Personalized PageRank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff. The key idea is to decouple the feature projection and message passing via a DP PageRank algorithm which learns the structure information and uses the top-$K$ neighbors determined by the PageRank for feature aggregation. By capturing the most important neighbors for each node and avoiding the layer-wise message passing, it bounds the node sensitivity and achieves improved privacy-utility tradeoff compared to layer-wise perturbation based methods. We theoretically analyze the node DP guarantee for the two processes combined together and empirically demonstrate better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.

翻译：图神经网络在基于图结构数据的学习中取得了巨大成功。然而，训练好的模型可能泄露图的敏感信息（包括节点特征和结构信息），由此引发了隐私担忧。本文旨在实现图神经网络训练中的节点级差分隐私，以保护节点及其连边。由于所有直接和多跳邻居通过逐层消息传递参与每个节点的梯度计算，且节点可能拥有的直接和多跳邻居数量无上限，现有差分隐私方法会因高节点灵敏度导致高昂的隐私成本或低效用。我们提出一种基于差分隐私近似个性化PageRank的解耦图神经网络，用于提高隐私-效用权衡下训图神经网络。核心思想是通过差分隐私PageRank算法解耦特征投影与消息传递：该算法学习结构信息，并利用PageRank确定的前K个邻居进行特征聚合。通过捕获每个节点的最重要邻居并避免逐层消息传递，该方法约束了节点灵敏度，相比基于逐层扰动的方案实现了更优的隐私-效用权衡。我们从理论上分析了两个过程联合的节点差分隐私保证，并通过实验证明在相同节点差分隐私等级下，DPAR相比现有最优方法具有更优的效用性。