Federated Learning (FL) is a distributed machine learning paradigm that allows clients to train models on their data while preserving their privacy. FL algorithms, such as Federated Averaging (FedAvg) and its variants, have been shown to converge well in many scenarios. However, these methods require clients to upload their local updates to the server in a synchronous manner, which can be slow and unreliable in realistic FL settings. To address this issue, researchers have developed asynchronous FL methods that allow clients to continue training on their local data using a stale global model. However, most of these methods simply aggregate all of the received updates without considering their relative contributions, which can slow down convergence. In this paper, we propose a contribution-aware asynchronous FL method that takes into account the staleness and statistical heterogeneity of the received updates. Our method dynamically adjusts the contribution of each update based on these factors, which can speed up convergence compared to existing methods.
翻译:联邦学习是一种分布式机器学习范式,允许客户端在保护隐私的前提下利用本地数据训练模型。联邦平均及其变体等联邦学习算法已在许多场景下展现出良好的收敛性。然而,这些方法要求客户端以同步方式向服务器上传本地更新,在真实的联邦学习环境中可能既缓慢又不可靠。为解决这一问题,研究者开发了异步联邦学习方法,允许客户端使用陈旧全局模型继续训练本地数据。但现有方法大多仅简单聚合所有接收到的更新,未考虑其相对贡献,这可能导致收敛速度变慢。本文提出一种贡献感知的异步联邦学习方法,该方法综合考虑接收更新的陈旧性与统计异质性,基于这些因素动态调整各更新的贡献权重,从而相比现有方法加速收敛过程。