Deep Neural Networks (DNNs) needs to be both efficient and robust for practical uses. Quantization and structure simplification are promising ways to adapt DNNs to mobile devices, and adversarial training is the most popular method to make DNNs robust. In this work, we try to obtain both features by applying a convergent relaxation quantization algorithm, Binary-Relax (BR), to a robust adversarial-trained model, ResNets Ensemble via Feynman-Kac Formalism (EnResNet). We also discover that high precision, such as ternary (tnn) and 4-bit, quantization will produce sparse DNNs. However, this sparsity is unstructured under advarsarial training. To solve the problems that adversarial training jeopardizes DNNs' accuracy on clean images and the struture of sparsity, we design a trade-off loss function that helps DNNs preserve their natural accuracy and improve the channel sparsity. With our trade-off loss function, we achieve both goals with no reduction of resistance under weak attacks and very minor reduction of resistance under strong attcks. Together with quantized EnResNet with trade-off loss function, we provide robust models that have high efficiency.

翻译：深度神经网络（DNN）在实际应用中需兼具高效性与鲁棒性。量化与结构简化是使DNN适配移动设备的有效途径，而对抗训练则是提升DNN鲁棒性的主流方法。本研究通过将收敛松弛量化算法Binary-Relax (BR)应用于基于Feynman-Kac形式论的鲁棒对抗训练模型ResNets Ensemble (EnResNet)，试图同时获得高效性与鲁棒性。我们进一步发现，高精度量化（如三值网络tnn和4位量化）会产生稀疏DNN，但这种稀疏性在对抗训练下呈现非结构化特征。针对对抗训练会降低DNN在干净图像上的准确率并破坏稀疏结构的问题，我们设计了一种权衡损失函数，帮助DNN保持原始准确率并提升通道稀疏性。借助该权衡损失函数，我们实现了在不降低弱攻击抵抗能力与极小幅度降低强攻击抵抗能力的前提下达成双重目标。结合采用权衡损失函数的量化EnResNet，我们最终提供了兼具高效性的鲁棒模型。