In federated learning (FL), clients cooperatively train a global model without revealing their raw data but gradients or parameters, while the local information can still be disclosed from local outputs transmitted to the parameter server. With such privacy concerns, a client may overly add artificial noise to his local updates to compromise the global model training, and we prove the selfish noise adding leads to an infinite price of anarchy (PoA). This paper proposes a novel pricing mechanism to regulate privacy-sensitive clients without verifying their parameter updates, unlike existing privacy mechanisms that assume the server's full knowledge of added noise. Without knowing the ground truth, our mechanism reaches the social optimum to best balance the global training error and privacy loss, according to the difference between a client's updated parameter and all clients' average parameter. We also improve the FL convergence bound by refining the aggregation rule at the server to account for different clients' noise variances. Moreover, we extend our pricing scheme to fit incomplete information of clients' privacy sensitivities, ensuring their truthful type reporting and the system's ex-ante budget balance. Simulations show that our pricing scheme greatly improves the system performance especially when clients have diverse privacy sensitivities.
翻译:在联邦学习中,客户端无需泄露原始数据,仅通过梯度或参数协作训练全局模型,但传输至参数服务器的本地输出仍可能泄露隐私信息。出于隐私顾虑,客户端可能过度向本地更新添加人工噪声,这会损害全局模型训练,我们证明自私的噪声添加行为将导致无政府状态价格(PoA)趋于无穷大。本文提出一种新颖的定价机制,用于调控对隐私敏感的客户端,且无需验证其参数更新——这与现有依赖服务器完全知晓添加噪声量的隐私机制不同。在未知真实值的情况下,本机制可根据客户端更新参数与全体客户端平均参数之间的差异,在全局训练误差与隐私损失之间实现社会最优平衡。通过改进服务器端的聚合规则以考虑不同客户端的噪声方差,我们还提升了联邦学习的收敛界。此外,我们将定价方案扩展至适应客户端隐私敏感度的不完全信息情形,确保其类型真实报告与系统的事前预算平衡。仿真结果表明,本定价方案显著提升了系统性能,尤其在客户端隐私敏感度存在差异时效果更为突出。