Federated Learning (FL) is a distributed machine learning approach that enables training on decentralized data while preserving privacy. However, FL systems often involve resource-constrained client devices with limited computational power, memory, storage, and bandwidth. This paper introduces FedMap, a novel method that aims to enhance the communication efficiency of FL deployments by collaboratively learning an increasingly sparse global model through iterative, unstructured pruning. Importantly, FedMap trains a global model from scratch, unlike other methods reported in the literature, making it ideal for privacy-critical use cases such as in the medical and finance domains, where suitable pre-training data is often limited. FedMap adapts iterative magnitude-based pruning to the FL setting, ensuring all clients prune and refine the same subset of the global model parameters, therefore gradually reducing the global model size and communication overhead. The iterative nature of FedMap, forming subsequent models as subsets of predecessors, avoids parameter reactivation issues seen in prior work, resulting in stable performance. In this paper we provide an extensive evaluation of FedMap across diverse settings, datasets, model architectures, and hyperparameters, assessing performance in both IID and non-IID environments. Comparative analysis against the baseline approach demonstrates FedMap's ability to achieve more stable client model performance. For IID scenarios, FedMap achieves over $90$\% pruning without significant performance degradation. In non-IID settings, it achieves at least $~80$\% pruning while maintaining accuracy. FedMap offers a promising solution to alleviate communication bottlenecks in FL systems while retaining model accuracy.

翻译：联邦学习（FL）是一种分布式机器学习方法，能够在保护隐私的前提下利用分散数据进行训练。然而，联邦学习系统通常涉及资源受限的客户端设备，其计算能力、内存、存储和带宽均有限。本文提出FedMap，这是一种旨在通过迭代式非结构化剪枝协同学习日益稀疏的全局模型，从而提升联邦学习部署通信效率的新方法。值得注意的是，与文献中报道的其他方法不同，FedMap从头开始训练全局模型，这使其特别适用于医疗和金融等隐私敏感领域——这些领域通常缺乏合适的预训练数据。FedMap将基于幅度的迭代剪枝技术适配到联邦学习场景中，确保所有客户端对全局模型参数的同一子集进行剪枝和优化，从而逐步减小全局模型规模并降低通信开销。FedMap的迭代特性使后续模型始终作为前驱模型的子集，避免了已有研究中出现的参数重新激活问题，从而获得稳定的性能表现。本文通过多种实验设置、数据集、模型架构和超参数对FedMap进行全面评估，并在独立同分布与非独立同分布环境中分析其性能。与基线方法的对比分析表明，FedMap能够实现更稳定的客户端模型性能。在独立同分布场景下，FedMap可实现超过$90$%的剪枝率且性能无显著下降；在非独立同分布设置中，其至少能达到约$~80$%的剪枝率同时保持准确率。FedMap为缓解联邦学习系统的通信瓶颈同时保持模型精度提供了一种具有前景的解决方案。