Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to conventional ML pipelines, the client local optimization and server aggregation procedure in FL are sensitive to the hyperparameter (HP) selection. Despite extensive research on tuning HPs for centralized ML, these methods yield suboptimal results when employed in FL. This is mainly because their "training-after-tuning" framework is unsuitable for FL with limited client computation power. While some approaches have been proposed for HP-Tuning in FL, they are limited to the HPs for client local updates. In this work, we propose a novel HP-tuning algorithm, called Federated Population-based Hyperparameter Tuning (FedPop), to address this vital yet challenging problem. FedPop employs population-based evolutionary algorithms to optimize the HPs, which accommodates various HP types at both the client and server sides. Compared with prior tuning methods, FedPop employs an online "tuning-while-training" framework, offering computational efficiency and enabling the exploration of a broader HP search space. Our empirical validation on the common FL benchmarks and complex real-world FL datasets, including full-sized Non-IID ImageNet-1K, demonstrates the effectiveness of the proposed method, which substantially outperforms the concurrent state-of-the-art HP-tuning methods in FL.

翻译：联邦学习（FL）是一种分布式机器学习（ML）范式，多个客户端无需集中其本地数据即可协作训练ML模型。与传统ML流程类似，FL中的客户端本地优化和服务器聚合过程对超参数（HP）的选择十分敏感。尽管针对集中式ML的超参数调优已有广泛研究，但这些方法在应用于FL时效果欠佳。这主要是因为其“先调优后训练”的框架不适用于客户端计算能力有限的FL场景。虽然已有一些针对FL超参数调优的方法被提出，但它们仅限于客户端本地更新的超参数。本文提出一种新颖的超参数调优算法——基于种群的联邦超参数调优（FedPop），以解决这一重要且具有挑战性的问题。FedPop采用基于种群的进化算法优化超参数，可同时适配客户端和服务器端的各类超参数。与现有调优方法相比，FedPop采用在线“边训练边调优”框架，具有计算高效性，并能探索更广阔的超参数搜索空间。我们在常见FL基准测试及复杂真实世界FL数据集（包括完整规模的非独立同分布ImageNet-1K）上进行的实证验证表明，所提方法显著优于当前FL领域最先进的超参数调优方法。