Mixed precision quantization has become an important technique for optimizing the execution of deep neural networks (DNNs). Certified robustness, which provides provable guarantees about a model's ability to withstand different adversarial perturbations, has rarely been addressed in quantization due to unacceptably high cost of certifying robustness. This paper introduces ARQ, an innovative mixed-precision quantization method that not only preserves the clean accuracy of the smoothed classifiers but also maintains their certified robustness. ARQ uses reinforcement learning to find accurate and robust DNN quantization, while efficiently leveraging randomized smoothing, a popular class of statistical DNN verification algorithms. ARQ consistently performs better than multiple state-of-the-art quantization techniques across all the benchmarks and the input perturbation levels. The performance of ARQ quantized networks reaches that of the original DNN with floating-point weights, but with only 1.5% instructions and the highest certified radius. ARQ code is available at https://anonymous.4open.science/r/ARQ-FE4B.
翻译:混合精度量化已成为优化深度神经网络(DNN)执行的重要技术。可认证鲁棒性为模型抵御不同对抗性扰动的能力提供了可证明的保证,但由于认证鲁棒性的成本过高,该特性在量化领域很少被涉及。本文介绍了ARQ,一种创新的混合精度量化方法,该方法不仅能保持平滑分类器的原始精度,还能维持其可认证鲁棒性。ARQ利用强化学习来寻找精确且鲁棒的DNN量化方案,同时高效地利用了随机平滑——一类流行的统计DNN验证算法。在所有基准测试和输入扰动水平下,ARQ的表现始终优于多种最先进的量化技术。ARQ量化网络的性能达到了原始浮点权重DNN的水平,但仅需1.5%的指令量,并实现了最高的可认证半径。ARQ代码可在 https://anonymous.4open.science/r/ARQ-FE4B 获取。