Reducing the memory footprint of Machine Learning (ML) models, particularly Deep Neural Networks (DNNs), is essential to enable their deployment into resource-constrained tiny devices. However, a disadvantage of DNN models is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. Therefore, the challenge is how to create accurate, robust, and tiny DNN models deployable on resource-constrained embedded devices. This paper reports the results of devising a tiny DNN model, robust to adversarial black and white box attacks, trained with an automatic quantizationaware training framework, i.e. QKeras, with deep quantization loss accounted in the learning loop, thereby making the designed DNNs more accurate for deployment on tiny devices. We investigated how QKeras and an adversarial robustness technique, Jacobian Regularization (JR), can provide a co-optimization strategy by exploiting the DNN topology and the per layer JR approach to produce robust yet tiny deeply quantized DNN models. As a result, a new DNN model implementing this cooptimization strategy was conceived, developed and tested on three datasets containing both images and audio inputs, as well as compared its performance with existing benchmarks against various white-box and black-box attacks. Experimental results demonstrated that on average our proposed DNN model resulted in 8.3% and 79.5% higher accuracy than MLCommons/Tiny benchmarks in the presence of white-box and black-box attacks on the CIFAR-10 image dataset and a subset of the Google Speech Commands audio dataset respectively. It was also 6.5% more accurate for black-box attacks on the SVHN image dataset.

翻译：减少机器学习模型（特别是深度神经网络）的内存占用，对于将其部署到资源受限的微型设备至关重要。然而，深度神经网络模型的缺陷在于其对对抗攻击的脆弱性——通过在输入中添加微小扰动即可欺骗模型。因此，如何在资源受限的嵌入式设备上创建准确、鲁棒且微型化的深度神经网络模型成为关键挑战。本文报告了一种微型深度神经网络模型的构建方法，该模型对黑盒与白盒对抗攻击具有鲁棒性，采用自动量化感知训练框架（即QKeras）进行训练，并在学习循环中计入深度量化损失，从而提升所设计深度神经网络在微型设备上的部署精度。我们研究了如何利用QKeras与对抗鲁棒性技术——雅可比正则化（JR）形成协同优化策略：通过结合深度神经网络拓扑结构与逐层JR方法，生成鲁棒且微型化的深度量化模型。最终，我们设计并实现了一种采用该协同优化策略的新型深度神经网络模型，在包含图像与音频输入的三类数据集上进行了测试，并与现有基准模型在白盒与黑盒攻击下的性能进行了对比。实验结果表明，在CIFAR-10图像数据集上面对白盒攻击时，以及Google语音命令音频数据子集上面对黑盒攻击时，我们提出的深度神经网络模型平均准确率分别比MLCommons/Tiny基准高8.3%和79.5%；在SVHN图像数据集的黑盒攻击场景下，准确率也提升了6.5%。