Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to conventional ML pipelines, the client local optimization and server aggregation procedure in FL are sensitive to the hyperparameter (HP) selection. Despite extensive research on tuning HPs for centralized ML, these methods yield suboptimal results when employed in FL. This is mainly because their "training-after-tuning" framework is unsuitable for FL with limited client computation power. While some approaches have been proposed for HP-Tuning in FL, they are limited to the HPs for client local updates. In this work, we propose a novel HP-tuning algorithm, called Federated Population-based Hyperparameter Tuning (FedPop), to address this vital yet challenging problem. FedPop employs population-based evolutionary algorithms to optimize the HPs, which accommodates various HP types at both client and server sides. Compared with prior tuning methods, FedPop employs an online "tuning-while-training" framework, offering computational efficiency and enabling the exploration of a broader HP search space. Our empirical validation on the common FL benchmarks and complex real-world FL datasets demonstrates the effectiveness of the proposed method, which substantially outperforms the concurrent state-of-the-art HP tuning methods for FL.

翻译：联邦学习（FL）是一种分布式机器学习（ML）范式，其中多个客户端在不集中本地数据的情况下协作训练ML模型。与传统的ML流程类似，FL中的客户端本地优化和服务器聚合过程对超参数（HP）选择高度敏感。尽管针对集中式ML的超参数调优已有广泛研究，但这些方法在应用于FL时效果欠佳，主要原因在于其"先调优后训练"框架不适用于客户端计算能力受限的FL场景。虽然目前已有针对FL中超参数调优的若干方法，但它们仅限于客户端本地更新的超参数。本文提出一种新颖的超参数调优算法——联邦化基于种群的超参数调优（FedPop），以应对这一关键且具有挑战性的问题。FedPop采用基于种群的进化算法优化超参数，可同时适配客户端和服务端的多种超参数类型。与先前的调优方法相比，FedPop采用在线"边训练边调优"框架，不仅具备计算效率优势，还能探索更广泛的超参数搜索空间。我们在常见FL基准测试及复杂真实世界FL数据集上的实证验证表明，所提方法具有显著有效性，其性能大幅优于当前最先进的FL超参数调优方法。