As a popular channel pruning method for convolutional neural networks (CNNs), network slimming (NS) has a three-stage process: (1) it trains a CNN with $\ell_1$ regularization applied to the scaling factors of the batch normalization layers; (2) it removes channels whose scaling factors are below a chosen threshold; and (3) it retrains the pruned model to recover the original accuracy. This time-consuming, three-step process is a result of using subgradient descent to train CNNs. Because subgradient descent does not exactly train CNNs towards sparse, accurate structures, the latter two steps are necessary. Moreover, subgradient descent does not have any convergence guarantee. Therefore, we develop an alternative algorithm called proximal NS. Our proposed algorithm trains CNNs towards sparse, accurate structures, so identifying a scaling factor threshold is unnecessary and fine tuning the pruned CNNs is optional. Using Kurdyka-{\L}ojasiewicz assumptions, we establish global convergence of proximal NS. Lastly, we validate the efficacy of the proposed algorithm on VGGNet, DenseNet and ResNet on CIFAR 10/100. Our experiments demonstrate that after one round of training, proximal NS yields a CNN with competitive accuracy and compression.

翻译：作为卷积神经网络（CNN）中一种流行的通道剪枝方法，网络剪枝（NS）包含三个阶段：（1）对批量归一化层的缩放因子施加ℓ₁正则化训练CNN；（2）移除缩放因子低于选定阈值的通道；（3）对剪枝后的模型进行再训练以恢复原始精度。这一耗时的三步流程源于使用次梯度下降法训练CNN。由于次梯度下降法无法精确训练CNN获得稀疏且准确的结构，后两个步骤是必要的。此外，次梯度下降法缺乏收敛性保证。为此，我们提出了一种替代算法——近端NS。该算法旨在训练CNN获得稀疏且准确的结构，因此无需确定缩放因子阈值，且对剪枝后模型的微调是可选的。基于Kurdyka-Łojasiewicz假设，我们证明了近端NS的全局收敛性。最后，在CIFAR-10/100数据集上针对VGGNet、DenseNet和ResNet的验证表明：经过一轮训练，近端NS即可获得具有竞争性精度和压缩率的CNN。