Compared with full client participation, partial client participation is a more practical scenario in federated learning, but it may amplify some challenges in federated learning, such as data heterogeneity. The lack of inactive clients' updates in partial client participation makes it more likely for the model aggregation to deviate from the aggregation based on full client participation. Training with large batches on individual clients is proposed to address data heterogeneity in general, but their effectiveness under partial client participation is not clear. Motivated by these challenges, we propose to develop a novel federated learning framework, referred to as FedAMD, for partial client participation. The core idea is anchor sampling, which separates partial participants into anchor and miner groups. Each client in the anchor group aims at the local bullseye with the gradient computation using a large batch. Guided by the bullseyes, clients in the miner group steer multiple near-optimal local updates using small batches and update the global model. By integrating the results of the two groups, FedAMD is able to accelerate the training process and improve the model performance. Measured by $\epsilon$-approximation and compared to the state-of-the-art methods, FedAMD achieves the convergence by up to $O(1/\epsilon)$ fewer communication rounds under non-convex objectives. Empirical studies on real-world datasets validate the effectiveness of FedAMD and demonstrate the superiority of the proposed algorithm: Not only does it considerably save computation and communication costs, but also the test accuracy significantly improves.

翻译：相比全客户端参与，部分客户端参与是联邦学习中更实际的场景，但其可能加剧联邦学习中的某些挑战（如数据异质性）。部分客户端参与中缺失非活跃客户端的更新，使得模型聚合更容易偏离基于全客户端参与的聚合。针对数据异质性，现有研究通常建议在单个客户端上使用大批量训练，但该方法在部分客户端参与场景下的有效性尚不明确。受这些挑战驱动，我们提出一种新颖的联邦学习框架FedAMD（Federated Learning with Anchor and Miner Division），专为部分客户端参与设计。其核心思想是锚定采样（Anchor Sampling），将部分参与者分为锚定组（Anchor Group）和挖掘组（Miner Group）。锚定组中的每个客户端通过大批量梯度计算瞄准局部目标（Local Bullseye），而挖掘组客户端则以这些目标为导向，使用小批量执行多个近最优的局部更新，并更新全局模型。通过整合两组结果，FedAMD能够加速训练过程并提升模型性能。基于$\epsilon$-近似度量并与现有最优方法对比，FedAMD在非凸目标下可减少多达$O(1/\epsilon)$的通信轮次实现收敛。在真实数据集上的实验验证了FedAMD的有效性，并展示了所提算法的优越性：不仅显著降低计算与通信成本，还大幅提升测试精度。