Structural pruning has become an integral part of neural network optimization, used to achieve architectural configurations which can be deployed and run more efficiently on embedded devices. Previous results showed that pruning is possible with minimum performance loss by utilizing a reinforcement learning agent which makes decisions about the sparsity level of each neural layer by maximizing as a reward the accuracy of the network. We introduce a novel information-theoretic reward function which minimizes the spatial entropy of convolutional activations. This minimization ultimately acts as a proxy for maintaining accuracy, although these two criteria are not related in any way. Our method shows that there is another possibility to preserve accuracy without the need to directly optimize it in the agent's reward function. In our experiments, we were able to reduce the total number of FLOPS of multiple popular neural network architectures by 5-10x, incurring minimal or no performance drop and being on par with the solution found by maximizing the accuracy.

翻译：结构化剪枝已成为神经网络优化中的重要组成部分，用于实现可在嵌入式设备上更高效部署和运行的架构配置。以往研究表明，通过利用强化学习代理对每个神经层的稀疏度进行决策，并以网络精度作为奖励最大化目标，可在性能损失最小的情况下实现剪枝。我们提出了一种新颖的信息论奖励函数，用于最小化卷积激活的空间熵。尽管这两个标准并无直接关联，但该最小化最终可作为保持精度的代理指标。我们的方法表明，无需在代理奖励函数中直接优化精度，也能找到另一种保持精度的方法。在实验中，我们能够将多种主流神经网络架构的总浮点运算量（FLOPS）降低5-10倍，同时仅产生极小或零性能损失，其效果与通过最大化精度找到的解决方案相当。