Network quantization is a dominant paradigm of model compression. However, the abrupt changes in quantized weights during training often lead to severe loss fluctuations and result in a sharp loss landscape, making the gradients unstable and thus degrading the performance. Recently, Sharpness-Aware Minimization (SAM) has been proposed to smooth the loss landscape and improve the generalization performance of the models. Nevertheless, directly applying SAM to the quantized models can lead to perturbation mismatch or diminishment issues, resulting in suboptimal performance. In this paper, we propose a novel method, dubbed Sharpness-Aware Quantization (SAQ), to explore the effect of SAM in model compression, particularly quantization for the first time. Specifically, we first provide a unified view of quantization and SAM by treating them as introducing quantization noises and adversarial perturbations to the model weights, respectively. According to whether the noise and perturbation terms depend on each other, SAQ can be formulated into three cases, which are analyzed and compared comprehensively. Furthermore, by introducing an efficient training strategy, SAQ only incurs a little additional training overhead compared with the default optimizer (e.g., SGD or AdamW). Extensive experiments on both convolutional neural networks and Transformers across various datasets (i.e., ImageNet, CIFAR-10/100, Oxford Flowers-102, Oxford-IIIT Pets) show that SAQ improves the generalization performance of the quantized models, yielding the SOTA results in uniform quantization. For example, on ImageNet, SAQ outperforms AdamW by 1.2% on the Top-1 accuracy for 4-bit ViT-B/16. Our 4-bit ResNet-50 surpasses the previous SOTA method by 0.9% on the Top-1 accuracy.

翻译：网络量化是模型压缩的主要范式之一。然而，训练过程中量化权重的突变常导致严重的损失波动，形成尖锐的损失景观，使梯度不稳定并降低模型性能。近期提出的锐度感知最小化方法通过平滑损失景观来提升模型泛化性能。但直接将SAM应用于量化模型会导致扰动不匹配或衰减问题，从而产生次优效果。本文首次提出名为锐度感知量化的新方法，探索SAM在模型压缩（特别是量化）中的作用。具体而言，我们首先将量化和SAM分别视为向模型权重注入量化噪声与对抗扰动，建立统一视角。根据噪声项与扰动项是否相互依赖，SAQ可划分为三种情形，并对其进行全面分析与比较。此外，通过引入高效训练策略，与默认优化器（如SGD或AdamW）相比，SAQ仅增加少量额外训练开销。在包含卷积神经网络和Transformer的多种数据集（ImageNet、CIFAR-10/100、Oxford Flowers-102、Oxford-IIIT Pets）上的大量实验表明，SAQ可提升量化模型的泛化性能，在均匀量化中取得最先进结果。例如，在ImageNet上，SAQ在4比特ViT-B/16的Top-1准确率上较AdamW提升1.2%。我们的4比特ResNet-50在Top-1准确率上超越此前最优方法0.9%。