With the remarkable success of deep learning recently, efficient network compression algorithms are urgently demanded for releasing the potential computational power of edge devices, such as smartphones or tablets. However, optimal network pruning is a non-trivial task which mathematically is an NP-hard problem. Previous researchers explain training a pruned network as buying a lottery ticket. In this paper, we investigate the Magnitude-Based Pruning (MBP) scheme and analyze it from a novel perspective through Fourier analysis on the deep learning model to guide model designation. Besides explaining the generalization ability of MBP using Fourier transform, we also propose a novel two-stage pruning approach, where one stage is to obtain the topological structure of the pruned network and the other stage is to retrain the pruned network to recover the capacity using knowledge distillation from lower to higher on the frequency domain. Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate the superiority of our novel Fourier analysis based MBP compared to other traditional MBP algorithms.

翻译：最近深层学习取得了显著的成功,因此迫切需要高效率的网络压缩算法,以释放边缘设备(如智能手机或平板电脑)的潜在计算能力。然而,最佳的网络运行是一种非三阶段性的任务,从数学上讲,这是一个NP-硬问题。以前的研究人员解释说,训练一个经切割的网络是为了购买彩票。在本文中,我们研究了磁基-普鲁宁(MBP)计划,并通过Fourier对深层学习模型的分析,从新颖的角度分析它,以指导模型的指定。除了用Fourier变换解释MBP(MBP)的通用能力外,我们还提出了一个新型的两阶段运行方法,即一个阶段是获得经切割的网络的顶层结构,另一个阶段是重新培训经切割的网络,以便利用频率领域从低到高的知识蒸馏恢复能力。关于CFAR-10和CIFAR-100的大规模实验表明我们基于MBP和其他传统的MBP算法的Fourier分析具有优越性。