Neural network pruning has traditionally focused on weight-based criteria to achieve model compression, frequently overlooking the crucial balance between adversarial robustness and accuracy. Existing approaches often fail to preserve robustness in pruned networks, leaving them more susceptible to adversarial attacks. This paper introduces Module Robustness Sensitivity (MRS), a novel metric that quantifies layer-wise sensitivity to adversarial perturbations and dynamically informs pruning decisions. Leveraging MRS, we propose Module Robust Pruning and Fine-Tuning (MRPF), an adaptive pruning algorithm compatible with any adversarial training method, offering both flexibility and scalability. Extensive experiments on SVHN, CIFAR, and Tiny-ImageNet across diverse architectures, including ResNet, VGG, and MobileViT, demonstrate that MRPF significantly enhances adversarial robustness while maintaining competitive accuracy and computational efficiency. Furthermore, MRPF consistently outperforms state-of-the-art structured pruning methods in balancing robustness, accuracy, and compression. This work establishes a practical and generalizable framework for robust pruning, addressing the long-standing trade-off between model compression and robustness preservation.
翻译:传统神经网络剪枝方法主要依据权重准则实现模型压缩,常忽视对抗鲁棒性与准确性之间的关键平衡。现有方法往往难以在剪枝后网络中保持鲁棒性,导致其更易受对抗攻击影响。本文提出模块鲁棒性敏感度(MRS)这一新颖指标,该指标能量化网络层对对抗扰动的敏感性,并动态指导剪枝决策。基于MRS,我们提出模块鲁棒剪枝与微调(MRPF)算法——一种兼容任意对抗训练方法的自适应剪枝算法,兼具灵活性与可扩展性。在SVHN、CIFAR和Tiny-ImageNet数据集上,针对ResNet、VGG和MobileViT等多种架构的广泛实验表明,MRPF在保持竞争力准确率与计算效率的同时,能显著提升对抗鲁棒性。此外,在平衡鲁棒性、准确率与压缩率方面,MRPF持续优于当前最先进的结构化剪枝方法。本研究为鲁棒剪枝建立了实用且可泛化的框架,解决了模型压缩与鲁棒性保持之间长期存在的权衡问题。