Federated optimization (FedOpt), which targets at collaboratively training a learning model across a large number of distributed clients, is vital for federated learning. The primary concerns in FedOpt can be attributed to the model divergence and communication efficiency, which significantly affect the performance. In this paper, we propose a new method, i.e., LoSAC, to learn from heterogeneous distributed data more efficiently. Its key algorithmic insight is to locally update the estimate for the global full gradient after {each} regular local model update. Thus, LoSAC can keep clients' information refreshed in a more compact way. In particular, we have studied the convergence result for LoSAC. Besides, the bonus of LoSAC is the ability to defend the information leakage from the recent technique Deep Leakage Gradients (DLG). Finally, experiments have verified the superiority of LoSAC comparing with state-of-the-art FedOpt algorithms. Specifically, LoSAC significantly improves communication efficiency by more than $100\%$ on average, mitigates the model divergence problem and equips with the defense ability against DLG.

翻译：联邦优化（FedOpt）旨在跨大量分布式客户端协同训练学习模型，对联邦学习至关重要。FedOpt中的主要问题可归因于模型发散和通信效率，这两者显著影响性能。本文提出了一种新方法——LoSAC，用于更高效地从异构分布式数据中学习。其核心算法思想是在每次常规局部模型更新后，局部更新全局全梯度的估计值。因此，LoSAC能以更紧凑的方式保持客户端信息的刷新。特别地，我们研究了LoSAC的收敛性结果。此外，LoSAC的额外优势在于能够防御近期技术“深层泄露梯度”（DLG）导致的信息泄露。最后，实验验证了LoSAC相较于最先进的FedOpt算法的优越性。具体而言，LoSAC平均将通信效率显著提升超过100%，缓解了模型发散问题，并具备针对DLG的防御能力。