Federated learning (FL) has prevailed as an efficient and privacy-preserved scheme for distributed learning. In this work, we mainly focus on the optimization of computation and communication in FL from a view of pruning. By adopting layer-wise pruning in local training and federated updating, we formulate an explicit FL pruning framework, FedLP (Federated Layer-wise Pruning), which is model-agnostic and universal for different types of deep learning models. Two specific schemes of FedLP are designed for scenarios with homogeneous local models and heterogeneous ones. Both theoretical and experimental evaluations are developed to verify that FedLP relieves the system bottlenecks of communication and computation with marginal performance decay. To the best of our knowledge, FedLP is the first framework that formally introduces the layer-wise pruning into FL. Within the scope of federated learning, more variants and combinations can be further designed based on FedLP.

翻译：联邦学习（FL）已成为分布式学习中高效且隐私保护的主流方案。本研究主要从剪枝视角出发，优化FL中的计算与通信效率。通过在本地训练和联邦更新中采用层级剪枝，我们构建了一个显式的FL剪枝框架FedLP（联邦层级剪枝），该框架具有模型无关性，可适用于不同类型的深度学习模型。针对同质本地模型与异质本地模型场景，我们设计了两种FedLP具体方案。理论与实验评估均验证了FedLP在仅造成轻微性能衰减的情况下，能有效缓解通信与计算的系统瓶颈。据我们所知，FedLP是首个将层级剪枝正式引入FL的框架。在联邦学习范畴内，可基于FedLP进一步设计更多变体与组合方案。