Asynchronous Federated Learning (AFL) has emerged as a significant research area in recent years. By not waiting for slower clients and executing the training process concurrently, it achieves faster training speed compared to traditional federated learning. However, due to the staleness introduced by the asynchronous process, its performance may degrade in some scenarios. Existing methods often use the round difference between the current model and the global model as the sole measure of staleness, which is coarse-grained and lacks observation of the model itself, thereby limiting the performance ceiling of asynchronous methods. In this paper, we propose FedPSA (Parameter Sensitivity-based Asynchronous Federated Learning), a more fine-grained AFL framework that leverages parameter sensitivity to measure model obsolescence and establishes a dynamic momentum queue to assess the current training phase in real time, thereby adjusting the tolerance for outdated information dynamically. Extensive experiments on multiple datasets and comparisons with various methods demonstrate the superior performance of FedPSA, achieving up to 6.37\% improvement over baseline methods and 1.93\% over the current state-of-the-art method.
翻译:近年来,异步联邦学习已成为一个重要的研究领域。它无需等待较慢的客户端,并能够并发执行训练过程,因此相比传统联邦学习实现了更快的训练速度。然而,由于异步过程引入的陈旧性,其性能在某些场景下可能会下降。现有方法通常仅使用当前模型与全局模型之间的轮次差异作为陈旧性的度量标准,这种度量方式较为粗粒度且缺乏对模型本身的观察,从而限制了异步方法的性能上限。本文提出FedPSA(基于参数敏感性的异步联邦学习),这是一个更细粒度的AFL框架。它利用参数敏感性来衡量模型陈旧性,并建立一个动态动量队列来实时评估当前训练阶段,从而动态调整对过时信息的容忍度。在多个数据集上进行的大量实验以及与多种方法的比较表明,FedPSA具有优越的性能,相较于基线方法最高可提升6.37%,相较于当前最先进方法可提升1.93%。