We propose a systematic analysis of deep neural networks (DNNs) based on a signal processing technique for network parameter removal, in the form of synaptic filters that identifies the fragility, robustness and antifragility characteristics of DNN parameters. Our proposed analysis investigates if the DNN performance is impacted negatively, invariantly, or positively on both clean and adversarially perturbed test datasets when the DNN undergoes synaptic filtering. We define three \textit{filtering scores} for quantifying the fragility, robustness and antifragility characteristics of DNN parameters based on the performances for (i) clean dataset, (ii) adversarial dataset, and (iii) the difference in performances of clean and adversarial datasets. We validate the proposed systematic analysis on ResNet-18, ResNet-50, SqueezeNet-v1.1 and ShuffleNet V2 x1.0 network architectures for MNIST, CIFAR10 and Tiny ImageNet datasets. The filtering scores, for a given network architecture, identify network parameters that are invariant in characteristics across different datasets over learning epochs. Vice-versa, for a given dataset, the filtering scores identify the parameters that are invariant in characteristics across different network architectures. We show that our synaptic filtering method improves the test accuracy of ResNet and ShuffleNet models on adversarial datasets when only the robust and antifragile parameters are selectively retrained at any given epoch, thus demonstrating applications of the proposed strategy in improving model robustness.

翻译：我们提出基于信号处理技术的网络参数移除方法，即突触滤波器，对深度神经网络(DNN)进行系统性分析，以识别DNN参数的脆弱性、鲁棒性与反脆弱性特征。本研究通过突触滤波处理，考察DNN在干净测试集与对抗扰动测试集上的性能是否呈现负向影响、保持不变或正向提升。我们定义了三种用于量化DNN参数特性的滤波分数：基于(1)干净数据集性能、(2)对抗数据集性能及(3)两者性能差异，分别对应脆弱性、鲁棒性与反脆弱性指标。在MNIST、CIFAR10与Tiny ImageNet数据集上，针对ResNet-18、ResNet-50、SqueezeNet-v1.1及ShuffleNet V2 x1.0网络架构进行了系统性分析验证。实验表明，对于特定网络架构，滤波分数能识别出跨不同训练周期与数据集呈现特性不变性的网络参数；反之，对于特定数据集，滤波分数可识别出跨不同网络架构保持特性不变性的参数。我们证明，当仅对鲁棒性与反脆弱性参数进行选择性重训练时，所提出的突触滤波方法能有效提升ResNet与ShuffleNet模型在对抗数据集上的测试准确率，从而展示该策略在增强模型鲁棒性方面的应用价值。