Federated Learning (FL) represents a growing machine learning (ML) paradigm designed for training models across numerous nodes that retain local datasets, all without directly exchanging the underlying private data with the parameter server (PS). Its increasing popularity is attributed to notable advantages in terms of training deep neural network (DNN) models under privacy aspects and efficient utilization of communication resources. Unfortunately, DNNs suffer from high computational and communication costs, as well as memory consumption in intricate tasks. These factors restrict the applicability of FL algorithms in communication-constrained systems with limited hardware resources. In this paper, we develop a novel algorithm that overcomes these limitations by synergistically combining a pruning-based method with the FL process, resulting in low-dimensional representations of the model with minimal communication cost, dubbed Masked Pruning over FL (MPFL). The algorithm operates by initially distributing weights to the nodes through the PS. Subsequently, each node locally trains its model and computes pruning masks. These low-dimensional masks are then transmitted back to the PS, which generates a consensus pruning mask, broadcasted back to the nodes. This iterative process enhances the robustness and stability of the masked pruning model. The generated mask is used to train the FL model, achieving significant bandwidth savings. We present an extensive experimental study demonstrating the superior performance of MPFL compared to existing methods. Additionally, we have developed an open-source software package for the benefit of researchers and developers in related fields.

翻译：联邦学习（FL）是一种新兴的机器学习（ML）范式，旨在跨多个保留本地数据集的数据节点训练模型，且无需直接与参数服务器（PS）交换底层私有数据。因在隐私保护及通信资源高效利用方面训练深度神经网络（DNN）模型具有显著优势，其应用日益广泛。然而，DNN在处理复杂任务时存在计算成本高、通信开销大及内存消耗高等问题，这些因素限制了FL算法在硬件资源受限的通信约束系统中的适用性。本文提出一种新颖算法，通过将剪枝方法与FL过程协同结合，以极低通信代价生成模型的低维表示（称为联邦学习掩码剪枝方法——MPFL）。该算法首先通过PS将权重分发给各节点，随后每个节点本地训练模型并计算剪枝掩码。这些低维掩码被回传至PS，由其生成共识剪枝掩码并广播回各节点。此迭代过程增强了掩码剪枝模型的鲁棒性与稳定性。生成的掩码用于训练FL模型，实现了显著的带宽节省。广泛的实验研究表明，MPFL相较于现有方法具有优越性能。此外，我们已开发开源软件包以惠及相关领域的研究人员与开发者。