Balancing convergence efficiency and robustness under Differential Privacy (DP) is a central challenge in Federated Learning (FL). While AdamW accelerates training and fine-tuning in large-scale models, we find that directly applying it to Differentially Private FL (DPFL) suffers from three major issues: (i) data heterogeneity and privacy noise jointly amplify the variance of second-moment estimator, (ii) DP perturbations bias the second-moment estimator, and (iii) DP amplify AdamW sensitivity to local overfitting, worsening client drift. We propose DP-FedAdamW, the first AdamW-based optimizer for DPFL. It restores AdamW under DP by stabilizing second-moment variance, removing DP-induced bias, and aligning local updates to the global descent to curb client drift. Theoretically, we establish an unbiased second-moment estimator and prove a linearly accelerated convergence rate without any heterogeneity assumption, while providing tighter $(\varepsilon,δ)$-DP guarantees. Our empirical results demonstrate the effectiveness of DP-FedAdamW across language and vision Transformers and ResNet-18. On Tiny-ImageNet (Swin-Base, $\varepsilon=1$), DP-FedAdamW outperforms the state-of-the-art (SOTA) by 5.83\%. The code is available in Appendix.

翻译：在差分隐私（DP）约束下平衡收敛效率与鲁棒性是联邦学习（FL）中的核心挑战。尽管AdamW在大规模模型的训练与微调中能加速进程，但我们发现将其直接应用于差分隐私联邦学习（DPFL）时存在三个主要问题：（i）数据异构性与隐私噪声共同放大了二阶矩估计量的方差；（ii）DP扰动导致二阶矩估计量产生偏差；（iii）DP放大了AdamW对局部过拟合的敏感性，加剧了客户端漂移。我们提出了DP-FedAdamW，这是首个基于AdamW的DPFL优化器。它通过稳定二阶矩方差、消除DP引入的偏差以及对齐局部更新与全局下降方向以抑制客户端漂移，从而在DP条件下恢复AdamW的性能。理论上，我们建立了无偏的二阶矩估计量，并在无需任何异构性假设的情况下证明了线性加速的收敛速率，同时提供了更紧的$(\varepsilon,\delta)$-DP保障。实验结果表明，DP-FedAdamW在语言与视觉Transformer及ResNet-18模型上均表现优异。在Tiny-ImageNet数据集（Swin-Base，$\varepsilon=1$）上，DP-FedAdamW以5.83%的优势超越了当前最优方法（SOTA）。代码已附录提供。