Recent advancements in federated learning (FL) have produced models that retain user privacy by training across multiple decentralized devices or systems holding local data samples. However, these strategies often neglect the inherent challenges of statistical heterogeneity and vulnerability to adversarial attacks, which can degrade model robustness and fairness. Personalized FL strategies offer some respite by adjusting models to fit individual client profiles, yet they tend to neglect server-side aggregation vulnerabilities. To address these issues, we propose Reinforcement Federated Learning (RFL), a novel framework that leverages deep reinforcement learning to adaptively optimize client contribution during aggregation, thereby enhancing both model robustness against malicious clients and fairness across participants under non-identically distributed settings. To achieve this goal, we propose a meticulous approach involving a Deep Deterministic Policy Gradient-based algorithm for continuous control of aggregation weights, an innovative client selection method based on model parameter distances, and a reward mechanism guided by validation set performance. Empirically, extensive experiments demonstrate that, in terms of robustness, RFL outperforms the state-of-the-art methods, while maintaining comparable levels of fairness, offering a promising solution to build resilient and fair federated systems.

翻译：联邦学习的最新进展已能够通过跨多个持有本地数据样本的分布式设备或系统进行训练，从而生成保留用户隐私的模型。然而，这些策略往往忽视了统计异质性和对抗性攻击的固有挑战，这可能导致模型鲁棒性和公平性的退化。个性化联邦学习策略通过调整模型以适应个体客户端特征提供了部分缓解方案，但常忽视服务器端聚合的脆弱性。为应对这些问题，我们提出强化联邦学习框架——一种利用深度强化学习自适应优化聚合过程中客户端贡献的新方法，从而在非独立同分布场景下增强对恶意客户端的模型鲁棒性及参与者间的公平性。为实现该目标，我们提出了精细化的方法：基于深度确定性策略梯度的聚合权重连续控制算法、基于模型参数距离的创新客户端选择机制，以及通过验证集性能引导的奖励函数。大量实验结果表明，在鲁棒性方面，RFL方法显著优于现有最先进方法，且能保持相当水平的公平性，为构建具有韧性且公平的联邦系统提供了可行解决方案。