Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to conventional ML pipelines, the client local optimization and server aggregation procedure in FL are sensitive to the hyperparameter (HP) selection. Despite extensive research on tuning HPs for centralized ML, these methods yield suboptimal results when employed in FL. This is mainly because their "training-after-tuning" framework is unsuitable for FL with limited client computation power. While some approaches have been proposed for HP-Tuning in FL, they are limited to the HPs for client local updates. In this work, we propose a novel HP-tuning algorithm, called Federated Population-based Hyperparameter Tuning (FedPop), to address this vital yet challenging problem. FedPop employs population-based evolutionary algorithms to optimize the HPs, which accommodates various HP types at both client and server sides. Compared with prior tuning methods, FedPop employs an online "tuning-while-training" framework, offering computational efficiency and enabling the exploration of a broader HP search space. Our empirical validation on the common FL benchmarks and complex real-world FL datasets demonstrates the effectiveness of the proposed method, which substantially outperforms the concurrent state-of-the-art HP tuning methods for FL.

翻译：联邦学习（Federated Learning, FL）是一种分布式机器学习范式，多个客户端无需集中其本地数据即可协作训练机器学习模型。与传统机器学习流程类似，联邦学习中的客户端本地优化与服务器聚合过程对超参数选择十分敏感。尽管针对集中式机器学习的超参数调优已有广泛研究，但这些方法应用于联邦学习时效果欠佳。这主要是因为其“先调优后训练”的框架不适用于客户端计算能力有限的联邦学习场景。虽然已有一些针对联邦学习超参数调优的方法被提出，但它们仅限于调整客户端本地更新的超参数。本工作提出一种新颖的超参数调优算法——基于联邦种群的超参数调优（FedPop），以解决这一重要而具挑战性的问题。FedPop采用基于种群的进化算法优化超参数，可同时处理客户端与服务器端的各类超参数。与现有调优方法相比，FedPop采用在线“边训练边调优”的框架，具有更高的计算效率，并能探索更广阔的超参数搜索空间。我们在常见联邦学习基准数据集及复杂的真实世界联邦学习数据集上进行了实证验证，结果表明所提方法显著优于当前联邦学习领域最先进的超参数调优方法。