It is necessary to develop efficient DNNs deployed on edge devices with limited computation resources. However, the compressed networks often execute new tasks in the target domain, which is different from the source domain where the original network is trained. It is important to investigate the robustness of compressed networks in two types of data distribution shifts: domain shifts and adversarial perturbations. In this study, we discover that compressed models are less robust to distribution shifts than their original networks. Interestingly, larger networks are more vulnerable to losing robustness than smaller ones, even when they are compressed to a similar size as the smaller networks. Furthermore, compact networks obtained by knowledge distillation are much more robust to distribution shifts than pruned networks. Finally, post-training quantization is a reliable method for achieving significant robustness to distribution shifts, and it outperforms both pruned and distilled models in terms of robustness.

翻译：在计算资源有限的边缘设备上部署高效的深度神经网络（DNN）是必要的。然而，压缩后的网络通常需要在目标域中执行新任务，而该目标域与原始网络训练的源域不同。因此，研究压缩网络在两种数据分布偏移（领域偏移和对抗扰动）下的鲁棒性至关重要。本研究发现，压缩模型对分布偏移的鲁棒性低于其原始网络。有趣的是，即使将大网络压缩至与小网络相近的尺寸，大网络在鲁棒性方面的损失仍比小网络更严重。此外，通过知识蒸馏获得的紧凑网络在分布偏移下的鲁棒性远优于剪枝网络。最后，训练后量化是一种可靠的方法，能够显著提升对分布偏移的鲁棒性，在鲁棒性方面优于剪枝模型和蒸馏模型。