Over the past few years, Federated Learning (FL) has become a popular distributed machine learning paradigm. FL involves a group of clients with decentralized data who collaborate to learn a common model under the coordination of a centralized server, with the goal of protecting clients' privacy by ensuring that local datasets never leave the clients and that the server only performs model aggregation. However, in realistic scenarios, the server may be able to collect a small amount of data that approximately mimics the population distribution and has stronger computational ability to perform the learning process. To address this, we focus on the hybrid FL framework in this paper. While previous hybrid FL work has shown that the alternative training of clients and server can increase convergence speed, it has focused on the scenario where clients fully participate and ignores the negative effect of partial participation. In this paper, we provide theoretical analysis of hybrid FL under clients' partial participation to validate that partial participation is the key constraint on convergence speed. We then propose a new algorithm called FedCLG, which investigates the two-fold role of the server in hybrid FL. Firstly, the server needs to process the training steps using its small amount of local datasets. Secondly, the server's calculated gradient needs to guide the participated clients' training and the server's aggregation. We validate our theoretical findings through numerical experiments, which show that our proposed method FedCLG outperforms state-of-the-art methods.

翻译：近年来，联邦学习已成为一种流行的分布式机器学习范式。联邦学习涉及一组拥有分散数据的客户端，它们在中央服务器的协调下协作学习一个通用模型，其目标是通过确保本地数据集始终留在客户端本地且服务器仅执行模型聚合来保护客户端隐私。然而，在实际场景中，服务器可能能够收集少量近似模拟总体分布的数据，并具备更强的计算能力来执行学习过程。针对这一问题，本文聚焦于混合联邦学习框架。尽管先前的混合联邦学习研究已表明客户端与服务器的交替训练可加快收敛速度，但这些工作主要关注客户端完全参与的场景，忽视了部分参与带来的负面影响。本文对客户端部分参与下的混合联邦学习进行了理论分析，验证了部分参与是收敛速度的关键制约因素。随后，我们提出了一种名为FedCLG的新算法，该算法探究了服务器在混合联邦学习中的双重作用：首先，服务器需要利用其少量本地数据集执行训练步骤；其次，服务器计算的梯度需指导参与客户端的训练以及服务器的聚合过程。通过数值实验验证了我们的理论发现，结果表明，所提出的FedCLG方法优于现有最优方法。