Federated learning (FL) promotes decentralized training while prioritizing data confidentiality. However, its application on resource-constrained devices is challenging due to the high demand for computation and memory resources to train deep learning models. Neural network pruning techniques, such as dynamic pruning, could enhance model efficiency, but directly adopting them in FL still poses substantial challenges, including post-pruning performance degradation, high activation memory usage, etc. To address these challenges, we propose FedMef, a novel and memory-efficient federated dynamic pruning framework. FedMef comprises two key components. First, we introduce the budget-aware extrusion that maintains pruning efficiency while preserving post-pruning performance by salvaging crucial information from parameters marked for pruning within a given budget. Second, we propose scaled activation pruning to effectively reduce activation memory footprints, which is particularly beneficial for deploying FL to memory-limited devices. Extensive experiments demonstrate the effectiveness of our proposed FedMef. In particular, it achieves a significant reduction of 28.5% in memory footprint compared to state-of-the-art methods while obtaining superior accuracy.

翻译：联邦学习（FL）在保障数据机密性的同时推动去中心化训练。然而，由于训练深度学习模型对计算和内存资源的高需求，其在资源受限设备上的应用面临挑战。神经网络剪枝技术（如动态剪枝）可提升模型效率，但直接在FL中采用仍存在显著挑战，包括剪枝后性能退化、激活内存占用过高等问题。针对这些挑战，我们提出FedMef——一种新颖且内存高效的联邦动态剪枝框架。FedMef包含两个关键组件。首先，我们引入预算感知型挤出技术，通过在给定预算内从标记为剪枝的参数中挽救关键信息，在维持剪枝效率的同时保留剪枝后性能。其次，我们提出缩放激活剪枝技术以有效降低激活内存占用，这对将FL部署到内存受限设备尤为有利。大量实验证明了所提FedMef的有效性。特别地，与现有最优方法相比，它在实现卓越准确率的同时将内存占用显著降低28.5%。