To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharp loss landscape and have poor weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with improved stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. To further reduce the magnitude of random noise while achieving better performance, we propose DP-FedSAM-$top_k$ by adopting the local update sparsification technique. From the theoretical perspective, we present the convergence analysis to investigate how our algorithms mitigate the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with R\'enyi DP, the sensitivity analysis of local updates, and generalization analysis. At last, we empirically confirm that our algorithms achieve state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.

翻译：为抵御联邦学习中的推理攻击并减轻敏感信息泄露，客户端级差分隐私联邦学习通过裁剪本地更新并添加随机噪声，已成为隐私保护的实际标准。然而，现有DPFL方法易导致尖锐的损失景观且权重扰动鲁棒性差，造成严重的性能退化。为解决这些问题，我们提出新型DPFL算法DP-FedSAM，该算法利用梯度扰动减轻差分隐私带来的负面影响。具体而言，DP-FedSAM集成锐度感知最小化优化器，生成具有更强稳定性与权重扰动鲁棒性的局部平坦模型，从而减小本地更新范数并增强对DP噪声的鲁棒性，进而提升性能。为在获得更优性能的同时进一步降低随机噪声幅度，我们采用本地更新稀疏化技术，提出DP-FedSAM-$top_k$算法。理论层面，我们通过收敛性分析探究算法如何缓解差分隐私导致的性能退化，同时基于Rényi差分隐私给出严格隐私保障、本地更新敏感性分析及泛化分析。实验结果表明，与现有DPFL领域最先进的基线方法相比，我们的算法达到了最优性能。