Mixed-precision quantization (MPQ) suffers from time-consuming policy search process (i.e., the bit-width assignment for each layer) on large-scale datasets (e.g., ISLVRC-2012), which heavily limits its practicability in real-world deployment scenarios. In this paper, we propose to search the effective MPQ policy by using a small proxy dataset for the model trained on a large-scale one. It breaks the routine that requires a consistent dataset at model training and MPQ policy search time, which can improve the MPQ searching efficiency significantly. However, the discrepant data distributions bring difficulties in searching for such a transferable MPQ policy. Motivated by the observation that quantization narrows the class margin and blurs the decision boundary, we search the policy that guarantees a general and dataset-independent property: discriminability of feature representations. Namely, we seek the policy that can robustly keep the intra-class compactness and inter-class separation. Our method offers several advantages, i.e., high proxy data utilization, no extra hyper-parameter tuning for approximating the relationship between full-precision and quantized model and high searching efficiency. We search high-quality MPQ policies with the proxy dataset that has only 4% of the data scale compared to the large-scale target dataset, achieving the same accuracy as searching directly on the latter, and improving the MPQ searching efficiency by up to 300 times.

翻译：混合精度量化（MPQ）在大规模数据集（如ISLVRC-2012）上进行策略搜索（即每层位宽分配）时耗时长，严重限制了其在实际部署场景中的实用性。本文提出利用小规模代理数据集为大规模数据集上训练的模型搜索有效的MPQ策略，打破了模型训练与MPQ策略搜索需使用一致数据集的常规，可显著提升MPQ搜索效率。然而，数据分布差异为搜索此类可迁移MPQ策略带来了困难。基于量化会缩小类别间隔并模糊决策边界的观察，我们搜索能保证通用且与数据集无关的特性——特征表征的可判别性的策略。即寻求能稳健保持类内紧凑性与类间分离性的策略。该方法具有多项优势：高代理数据利用率、无需额外超参数调优来近似全精度模型与量化模型的关系，以及高搜索效率。与大规模目标数据集相比，我们使用仅为其数据规模4%的代理数据集搜索到高质量MPQ策略，达到与直接在目标数据集上搜索相同的精度，并将MPQ搜索效率提升高达300倍。