Model quantization and compression is widely used techniques to reduce usage of computing resource at inference time. While state-of-the-art works have been achieved reasonable accuracy with higher bit such as 4bit or 8bit, but still it is challenging to quantize/compress a model further, e.g., 1bit or 2bit. To overcome the challenge, we focus on outliers in weights of a pre-trained model which disrupt effective lower bit quantization and compression. In this work, we propose Range Restriction Loss (R2-Loss) for building lower bit quantization and compression friendly models by removing outliers from weights during pre-training. By effectively restricting range of weights, we mold the overall distribution into a tight shape to ensure high quantization bit resolution, therefore allowing model compression and quantization techniques can to utilize their limited numeric representation powers better. We introduce three different, L-inf R2-Loss, its extension Margin R2-Loss and a new Soft-Min-MaxR2-Loss to be used as an auxiliary loss during full-precision model training. These R2-Loss can be used in different cases such as L-inf and Margin R2-Loss would be effective for symmetric quantization, while Soft-Min-Max R2-Loss shows better performance for model compression. In our experiment, R2-Loss improves lower bit quantization accuracy with state-of-the-art post-training quantization (PTQ), quantization-aware training (QAT), and model compression techniques. With R2-Loss, MobileNet-V2 2bit weight and 8bit activation PTQ, MobileNet-V1 2bit weight and activation QAT, ResNet18 1bit weight compression are improved to 59.49% from 50.66%, 59.05% from 55.96%, and 52.58% from 45.54%, respectively.

翻译：模型量化与压缩是推理时降低计算资源消耗的广泛技术。尽管现有最高水准方法在高比特（如4bit或8bit）下已取得合理精度，但进一步压缩/量化模型（例如1bit或2bit）仍具挑战性。为解决这一问题，我们聚焦于预训练模型中阻碍低位量化与压缩有效性的权重异常值。本文提出范围约束损失（R2-Loss），通过预训练过程中剔除权重异常值，构建有利于低位量化与压缩的模型。通过有效约束权重范围，我们将整体分布塑造成紧凑形态以保证高量化比特分辨率，从而使模型压缩与量化技术能更好地利用其有限的数值表示能力。我们引入三种不同的R2-Loss：L-inf R2-Loss、其扩展Margin R2-Loss以及新型Soft-Min-Max R2-Loss，作为全精度模型训练时的辅助损失函数。这些R2-Loss可应用于不同场景：L-inf与Margin R2-Loss对对称量化有效，而Soft-Min-Max R2-Loss在模型压缩中表现更优。实验中，R2-Loss显著提升了低位量化精度，涵盖最先进的训练后量化（PTQ）、量化感知训练（QAT）及模型压缩技术。采用R2-Loss后，MobileNet-V2的2bit权重与8bit激活PTQ从50.66%提升至59.49%，MobileNet-V1的2bit权重与激活QAT从55.96%提升至59.05%，ResNet18的1bit权重压缩从45.54%提升至52.58%。