Evolutionary Computation algorithms have been used to solve optimization problems in relation with architectural, hyper-parameter or training configuration, forging the field known today as Neural Architecture Search. These algorithms have been combined with other techniques such as the pruning of Neural Networks, which reduces the complexity of the network, and the Transfer Learning, which lets the import of knowledge from another problem related to the one at hand. The usage of several criteria to evaluate the quality of the evolutionary proposals is also a common case, in which the performance and complexity of the network are the most used criteria. This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. MO-EvoPruneDeepTL uses Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm, which guides the evolution based in the performance, complexity and robustness of the network, being the robustness a great quality indicator for the evolved models. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show that our proposal achieves promising results in all the objectives, and direct relation are presented among them. The experiments also show that the most influential neurons help us explain which parts of the input images are the most relevant for the prediction of the pruned neural network. Lastly, by virtue of the diversity within the Pareto front of pruning patterns produced by the proposal, it is shown that an ensemble of differently pruned models improves the overall performance and robustness of the trained networks.

翻译：进化计算算法已用于解决与架构、超参数或训练配置相关的优化问题，形成了如今所称的神经架构搜索领域。这些算法还与其他技术相结合，例如神经网络剪枝（可降低网络复杂度）和迁移学习（允许从当前问题的相关其他问题中导入知识）。在评估进化方案质量时，使用多个标准也是常见做法，其中网络性能和复杂度是最常用的标准。本文提出MO-EvoPruneDeepTL——一种多目标进化剪枝算法。MO-EvoPruneDeepTL利用迁移学习来适应深度神经网络的最后几层，将其替换为由遗传算法进化得到的稀疏层，该遗传算法基于网络性能、复杂度和鲁棒性指导进化过程，其中鲁棒性是评估进化模型质量的重要指标。我们使用多个数据集开展不同实验以评估本方案的益处。结果表明，本方案在所有目标上均取得了有前景的结果，且各目标之间呈现直接关联。实验还表明，最具影响力的神经元有助于解释输入图像的哪些部分对剪枝神经网络的预测最为关键。最后，得益于本方案产生的剪枝模式帕累托前沿的多样性，研究表明由不同剪枝模型组成的集成体可提升训练网络的整体性能和鲁棒性。