Federated learning (FL) has emerged as a promising distributed machine learning (ML) that enables collaborative model training across clients without exposing raw data, thereby preserving user privacy and reducing communication costs. Despite these benefits, traditional single-server FL suffers from high communication latency due to the aggregation of models from a large number of clients. While multi-server FL distributes workloads across edge servers, overlapping client coverage and uncoordinated selection often lead to resource contention, causing bandwidth conflicts and training failures. To address these limitations, we propose a decentralized reinforcement learning with conflict risk prediction, named RL CRP, to optimize client selection in multi-server FL systems. Specifically, each server estimates the likelihood of client selection conflicts using a categorical hidden Markov model based on its sparse historical client selection sequence. Then, a fairness-aware reward mechanism is incorporated to promote long-term client participation for minimizing training latency and resource contention. Extensive experiments demonstrate that the proposed RL-CRP framework effectively reduces inter-server conflicts and significantly improves training efficiency in terms of convergence speed and communication cost.

翻译：联邦学习（FL）作为一种有前景的分布式机器学习范式，能够在无需暴露原始数据的情况下实现跨客户端的协同模型训练，从而保护用户隐私并降低通信开销。尽管具有这些优势，传统的单服务器联邦学习由于需要聚合大量客户端的模型，存在通信延迟高的问题。虽然多服务器联邦学习将工作负载分布到多个边缘服务器，但客户端覆盖范围的重叠以及缺乏协调的选择策略常常导致资源争用，引发带宽冲突和训练失败。为应对这些局限性，我们提出了一种结合冲突风险预测的去中心化强化学习方法，命名为RL-CRP，以优化多服务器联邦学习系统中的客户端选择。具体而言，每个服务器基于其稀疏的历史客户端选择序列，利用分类隐马尔可夫模型来估计客户端选择冲突的可能性。随后，引入一种公平感知的奖励机制，以促进客户端的长期参与，从而最小化训练延迟和资源争用。大量实验表明，所提出的RL-CRP框架能有效减少服务器间的冲突，并在收敛速度和通信成本方面显著提升训练效率。