Existing Quantization-Aware Training (QAT) methods intensively depend on the complete labeled dataset or knowledge distillation to guarantee the performances toward Full Precision (FP) accuracies. However, empirical results show that QAT still has inferior results compared to its FP counterpart. One question is how to push QAT toward or even surpass FP performances. In this paper, we address this issue from a new perspective by injecting the vicinal data distribution information to improve the generalization performances of QAT effectively. We present a simple, novel, yet powerful method introducing an Consistency Regularization (CR) for QAT. Concretely, CR assumes that augmented samples should be consistent in the latent feature space. Our method generalizes well to different network architectures and various QAT methods. Extensive experiments demonstrate that our approach significantly outperforms the current state-of-the-art QAT methods and even FP counterparts.

翻译：现有的量化感知训练方法严重依赖完整的标注数据集或知识蒸馏来保证达到全精度精度。然而，实验结果表明，量化感知训练相比其全精度对应方法仍存在性能劣势。一个问题是如何推动量化感知训练达到甚至超越全精度性能。本文从一个新视角解决该问题，通过注入邻域数据分布信息有效提升量化感知训练的泛化性能。我们提出一种简单新颖且强大的方法，为量化感知训练引入一致性正则化。具体而言，一致性正则化假设增强样本应在潜在特征空间中保持一致。我们的方法能够良好泛化至不同网络架构及各类量化感知训练方法。大量实验表明，本方法显著优于当前最先进的量化感知训练方法，甚至超越全精度对应方法。