To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharp loss landscape and have poor weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with improved stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. To further reduce the magnitude of random noise while achieving better performance, we propose DP-FedSAM-$top_k$ by adopting the local update sparsification technique. From the theoretical perspective, we present the convergence analysis to investigate how our algorithms mitigate the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with R\'enyi DP, the sensitivity analysis of local updates, and generalization analysis. At last, we empirically confirm that our algorithms achieve state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.

翻译：为抵御联邦学习（FL）中的推理攻击并减少敏感信息泄露，客户端级差分隐私联邦学习（DPFL）通过裁剪局部更新并添加随机噪声，已成为隐私保护的事实标准。然而，现有DPFL方法往往导致损失景观陡峭且权重扰动鲁棒性较差，从而造成严重的性能退化。为缓解这些问题，我们提出一种名为DP-FedSAM的新型DPFL算法，该算法利用梯度扰动来减轻差分隐私（DP）带来的负面影响。具体而言，DP-FedSAM集成了锐度感知最小化（SAM）优化器，以生成具有更高稳定性和权重扰动鲁棒性的局部平坦模型，从而使得局部更新范数较小且对DP噪声具有鲁棒性，进而提升性能。为在实现更优性能的同时进一步降低随机噪声的幅度，我们提出DP-FedSAM-$top_k$，该算法采用局部更新稀疏化技术。从理论角度，我们给出收敛性分析以探究算法如何缓解由DP导致的性能退化。同时，我们基于Rényi差分隐私提供严格的隐私保证、局部更新的敏感性分析以及泛化分析。最后，实验证明我们的算法在与现有DPFL领域最先进（SOTA）基线方法相比时取得了最优性能。