Deep Neural Networks (DNNs) needs to be both efficient and robust for practical uses. Quantization and structure simplification are promising ways to adapt DNNs to mobile devices, and adversarial training is the most popular method to make DNNs robust. In this work, we try to obtain both features by applying a convergent relaxation quantization algorithm, Binary-Relax (BR), to a robust adversarial-trained model, ResNets Ensemble via Feynman-Kac Formalism (EnResNet). We also discover that high precision, such as ternary (tnn) and 4-bit, quantization will produce sparse DNNs. However, this sparsity is unstructured under advarsarial training. To solve the problems that adversarial training jeopardizes DNNs' accuracy on clean images and the struture of sparsity, we design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity. With our trade-off loss function, we achieve both goals with no reduction of resistance under weak attacks and very minor reduction of resistance under strong attcks. Together with quantized EnResNet with trade-off loss function, we provide robust models that have high efficiency.

翻译：深度神经网络（DNNs）在实际应用中需兼顾效率与鲁棒性。量化和结构简化是使DNNs适配移动设备的有效途径，而对抗训练则是提升DNNs鲁棒性的主流方法。本文通过将收敛松弛量化算法Binary-Relax（BR）应用于经过对抗训练的鲁棒模型——基于Feynman-Kac形式的ResNets集成（EnResNet），尝试同时获得高效性与鲁棒性。我们发现高精度量化（如三值量化tnn和4-bit量化）会产生稀疏DNNs，但这种稀疏性在对抗训练下呈现非结构化特征。为解决对抗训练损害DNNs对干净图像准确率以及破坏稀疏结构的问题，我们设计了一种权衡损失函数，帮助DNNs保留自然准确率并提升通道稀疏性。通过该权衡损失函数，我们在弱攻击下不降低抵抗能力，且在强攻击下仅极小程度降低抵抗能力的前提下同时实现了两个目标。结合应用权衡损失函数的量化EnResNet，我们提供了兼具高效率和鲁棒性的模型。