Evolutionary Computation algorithms have been used to solve optimization problems in relation with architectural, hyper-parameter or training configuration, forging the field known today as Neural Architecture Search. These algorithms have been combined with other techniques such as the pruning of Neural Networks, which reduces the complexity of the network, and the Transfer Learning, which lets the import of knowledge from another problem related to the one at hand. The usage of several criteria to evaluate the quality of the evolutionary proposals is also a common case, in which the performance and complexity of the network are the most used criteria. This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. \proposal uses Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm, which guides the evolution based in the performance, complexity and robustness of the network, being the robustness a great quality indicator for the evolved models. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show that our proposal achieves promising results in all the objectives, and direct relation are presented among them. The experiments also show that the most influential neurons help us explain which parts of the input images are the most relevant for the prediction of the pruned neural network. Lastly, by virtue of the diversity within the Pareto front of pruning patterns produced by the proposal, it is shown that an ensemble of differently pruned models improves the overall performance and robustness of the trained networks.

翻译：进化计算算法已被用于解决与架构、超参数或训练配置相关的优化问题，从而形成了当前所称的神经架构搜索领域。这些算法已与其他技术相结合，例如神经网络剪枝（可降低网络复杂度）和迁移学习（允许从与当前问题相关的其他问题中导入知识）。使用多个标准评估进化方案的质量也是常见情况，其中网络性能和复杂度是最常用的标准。本文提出MO-EvoPruneDeepTL算法，一种多目标进化剪枝算法。该算法利用迁移学习调整深度神经网络的最后几层，通过遗传算法进化出的稀疏层进行替换，遗传算法基于网络的性能、复杂度和鲁棒性指导进化过程，其中鲁棒性是评估进化模型质量的重要指标。我们使用多个数据集进行不同实验以评估所提方案的优越性。结果表明，我们的方案在所有目标上均取得了令人满意的结果，且各目标间存在直接关联。实验还表明，最具影响力的神经元有助于解释输入图像的哪些部分对剪枝后神经网络的预测最为关键。最后，得益于所提方案生成的帕累托剪枝模式前沿的多样性，实验证明由不同剪枝模型构成的集成方法能够显著提升训练网络的整体性能和鲁棒性。