Federated training of large deep neural networks can often be restrictive due to the increasing costs of communicating the updates with increasing model sizes. Various model pruning techniques have been designed in centralized settings to reduce inference times. Combining centralized pruning techniques with federated training seems intuitive for reducing communication costs -- by pruning the model parameters right before the communication step. Moreover, such a progressive model pruning approach during training can also reduce training times/costs. To this end, we propose FedSparsify, which performs model pruning during federated training. In our experiments in centralized and federated settings on the brain age prediction task (estimating a person's age from their brain MRI), we demonstrate that models can be pruned up to 95% sparsity without affecting performance even in challenging federated learning environments with highly heterogeneous data distributions. One surprising benefit of model pruning is improved model privacy. We demonstrate that models with high sparsity are less susceptible to membership inference attacks, a type of privacy attack.

翻译：大型深神经网络的联邦培训往往会受到限制,因为随着模型规模的扩大,通信更新的成本不断增加。在集中化环境下设计了各种模型修剪技术,以减少推断时间。将集中化修剪技术与联盟化培训相结合,对于降低通信成本似乎具有直觉性 -- -- 在通信步骤之前就修剪模型参数。此外,在培训过程中这种渐进型模型修剪方法也可以减少培训时间/成本。为此,我们提议美联储简化模型,在联合化培训期间进行模型修剪。在集中化和联合化环境中进行的关于大脑年龄预测任务的实验中(从大脑MRI中估计一个人的年龄),我们证明模型可以修补到95%的宽度,而不会影响在数据分布高度不一的充满差异的联邦化学习环境中的性能。模型修剪裁的一个令人惊讶的好处是改进模型隐私。我们证明,具有高度紧张性的模型的模型不易成为成员推论攻击的对象,一种隐私攻击的类型。