Deep learning harnesses massive parallel floating-point processing to train and evaluate large neural networks. Trends indicate that deeper and larger neural networks with an increasing number of parameters achieve higher accuracy than smaller neural networks. This performance improvement, which often requires heavy compute for both training and evaluation, eventually needs to translate well to resource-constrained hardware for practical value. Structured pruning asserts that while large networks enable us to find solutions to complex computer vision problems, a smaller, computationally efficient sub-network can be derived from the large neural network that retains model accuracy but significantly improves computational efficiency. We generalize structured pruning with algorithms for network augmentation, pruning, sub-network collapse and removal. In addition, we demonstrate efficient and stable convergence up to 93% sparsity and 95% FLOPs reduction without loss of inference accuracy using with continuous relaxation matching or exceeding the state of the art for all structured pruning methods. The resulting CNN executes efficiently on GPU hardware without computationally expensive sparse matrix operations. We achieve this with routine automatable operations on classification and segmentation problems using CIFAR-10, ImageNet, and CityScapes datasets with the ResNet and U-NET network architectures.

翻译：深度学习利用大规模并行浮点运算训练和评估大型神经网络。趋势表明，参数数量更深、更大的神经网络比小型神经网络能达到更高精度。这种性能提升通常需要大量计算资源用于训练和评估，最终需有效迁移至资源受限硬件才能实现实际价值。结构化剪枝理论认为，虽然大型网络能帮助解决复杂计算机视觉问题，但可从该大型神经网络中提取出保留模型精度但显著提升计算效率的小型计算高效子网络。我们通过网络增广、剪枝、子网络坍缩与移除算法对结构化剪枝进行泛化。此外，我们证明了使用连续松弛方法可实现高效稳定收敛，在达到93%稀疏度与95%浮点运算量缩减的同时保持推理精度，其性能超过或匹配所有结构化剪枝方法的现有最优水平。生成的卷积神经网络无需计算昂贵的稀疏矩阵运算即可在GPU硬件上高效执行。我们通过CIFAR-10、ImageNet和CityScapes数据集，基于ResNet和U-NET网络架构，对分类与分割问题实现了常规自动化操作流程。