Differentially private federated learning (DP-FL) is a promising technique for collaborative model training while ensuring provable privacy for clients. However, optimizing the tradeoff between privacy and accuracy remains a critical challenge. To our best knowledge, we propose the first DP-FL framework (namely UDP-FL), which universally harmonizes any randomization mechanism (e.g., an optimal one) with the Gaussian Moments Accountant (viz. DP-SGD) to significantly boost accuracy and convergence. Specifically, UDP-FL demonstrates enhanced model performance by mitigating the reliance on Gaussian noise. The key mediator variable in this transformation is the R\'enyi Differential Privacy notion, which is carefully used to harmonize privacy budgets. We also propose an innovative method to theoretically analyze the convergence for DP-FL (including our UDP-FL ) based on mode connectivity analysis. Moreover, we evaluate our UDP-FL through extensive experiments benchmarked against state-of-the-art (SOTA) methods, demonstrating superior performance on both privacy guarantees and model performance. Notably, UDP-FL exhibits substantial resilience against different inference attacks, indicating a significant advance in safeguarding sensitive data in federated learning environments.

翻译：差分隐私联邦学习（DP-FL）是一种在确保客户可证明隐私的前提下进行协同模型训练的前沿技术。然而，优化隐私与精度之间的权衡仍是一个关键挑战。据我们所知，我们首次提出了DP-FL框架（即UDP-FL），该框架通过将任意随机化机制（例如最优机制）与高斯矩会计方法（即DP-SGD）进行普适协调，显著提升了精度与收敛速度。具体而言，UDP-FL通过降低对高斯噪声的依赖，实现了模型性能的增强。该转换中的关键中介变量是R\'enyi差分隐私概念，其被精心用于协调隐私预算。我们还提出了一种基于模态连通性分析的理论方法，用于分析DP-FL（包括我们的UDP-FL）的收敛性。此外，我们通过大量实验评估了UDP-FL，并以最先进（SOTA）方法为基准进行对比，结果表明其在隐私保证和模型性能方面均具有优越性。值得注意的是，UDP-FL对不同推理攻击表现出显著的鲁棒性，这标志着在联邦学习环境中保护敏感数据方面取得了重要进展。