Federated Learning (FL) is a decentralized machine learning paradigm that enables collaborative model training across dispersed nodes without having to force individual nodes to share data. However, its broad adoption is hindered by the high communication costs of transmitting a large number of model parameters. This paper presents EvoFed, a novel approach that integrates Evolutionary Strategies (ES) with FL to address these challenges. EvoFed employs a concept of 'fitness-based information sharing', deviating significantly from the conventional model-based FL. Rather than exchanging the actual updated model parameters, each node transmits a distance-based similarity measure between the locally updated model and each member of the noise-perturbed model population. Each node, as well as the server, generates an identical population set of perturbed models in a completely synchronized fashion using the same random seeds. With properly chosen noise variance and population size, perturbed models can be combined to closely reflect the actual model updated using the local dataset, allowing the transmitted similarity measures (or fitness values) to carry nearly the complete information about the model parameters. As the population size is typically much smaller than the number of model parameters, the savings in communication load is large. The server aggregates these fitness values and is able to update the global model. This global fitness vector is then disseminated back to the nodes, each of which applies the same update to be synchronized to the global model. Our analysis shows that EvoFed converges, and our experimental results validate that at the cost of increased local processing loads, EvoFed achieves performance comparable to FedAvg while reducing overall communication requirements drastically in various practical settings.

翻译：联邦学习（FL）是一种去中心化的机器学习范式，能够在分散节点间实现协作模型训练，且无需强制各节点共享数据。然而，由于传输大量模型参数导致通信成本高昂，其广泛应用受到制约。本文提出EvoFed，一种将进化策略（ES）与联邦学习相结合的新方法，以应对上述挑战。EvoFed采用“基于适应度的信息共享”概念，显著偏离传统基于模型的联邦学习范式。各节点不再交换实际更新的模型参数，而是传输本地更新模型与噪声扰动模型种群中每个成员之间的基于距离的相似性度量。各节点及服务器利用相同的随机种子，以完全同步的方式生成相同的扰动模型种群。通过适当选择噪声方差与种群规模，扰动模型可组合后紧密反映使用本地数据集实际更新的模型，从而使传输的相似性度量（即适应度值）几乎携带模型参数的完整信息。由于种群规模通常远小于模型参数数量，通信负载的节省效果显著。服务器聚合这些适应度值并更新全局模型，随后将全局适应度向量分发回各节点，各节点应用相同更新以同步至全局模型。我们的分析表明EvoFed具有收敛性，实验结果验证了在增加本地处理负载的代价下，EvoFed能在多种实际场景中实现与FedAvg相当的性能，同时大幅降低总体通信需求。