This paper considers subject level privacy in the FL setting, where a subject is an individual whose private information is embodied by several data items either confined within a single federation user or distributed across multiple federation users. We propose two new algorithms that enforce subject level DP at each federation user locally. Our first algorithm, called LocalGroupDP, is a straightforward application of group differential privacy in the popular DP-SGD algorithm. Our second algorithm is based on a novel idea of hierarchical gradient averaging (HiGradAvgDP) for subjects participating in a training mini-batch. We also show that user level Local Differential Privacy (LDP) naturally guarantees subject level DP. We observe the problem of horizontal composition of subject level privacy loss in FL - subject level privacy loss incurred at individual users composes across the federation. We formally prove the subject level DP guarantee for our algorithms, and also show their effect on model utility loss. Our empirical evaluation on FEMNIST and Shakespeare datasets shows that LocalGroupDP delivers the best performance among our algorithms. However, its model utility lags behind that of models trained using a DP-SGD based algorithm that provides a weaker item level privacy guarantee. Privacy loss amplification due to subject sampling fractions and horizontal composition remain key challenges for model utility.

翻译：本文考虑了联邦学习场景中的主体级隐私保护问题，其中主体是指其隐私信息由多个数据项表征的个人，这些数据项可能集中在单个联邦用户内，也可能分布在多个联邦用户之间。我们提出了两种新算法，在本地联邦用户处强制执行主体级差分隐私。第一种算法称为LocalGroupDP，是组差分隐私在流行DP-SGD算法中的直接应用。第二种算法基于层级梯度平均（HiGradAvgDP）这一新颖思想，针对参与训练小批量的主体实施隐私保护。我们同时证明，用户级本地差分隐私（LDP）自然保证了主体级差分隐私。我们观察到联邦学习中主体级隐私损失的水平组合问题——单个用户产生的主体级隐私损失会在联邦范围内组合叠加。我们严格证明了所提算法的主体级差分隐私保证，并展示了它们对模型效用损失的影响。在FEMNIST和Shakespeare数据集上的实验评估表明，LocalGroupDP在我们的算法中取得了最佳性能。然而，其模型效用落后于采用提供较弱项目级隐私保证的DP-SGD算法训练的模型。由主体采样分数和水平组合导致的隐私损失放大仍然是模型效用的关键挑战。