Quantization is of significance for compressing the over-parameterized deep neural models and deploying them on resource-limited devices. Fixed-precision quantization suffers from performance drop due to the limited numerical representation ability. Conversely, mixed-precision quantization (MPQ) is advocated to compress the model effectively by allocating heterogeneous bit-width for layers. MPQ is typically organized into a searching-retraining two-stage process. Previous works only focus on determining the optimal bit-width configuration in the first stage efficiently, while ignoring the considerable time costs in the second stage. However, retraining always consumes hundreds of GPU-hours on the cutting-edge GPUs, thus hindering deployment efficiency significantly. In this paper, we devise a one-shot training-searching paradigm for mixed-precision model compression. Specifically, in the first stage, all potential bit-width configurations are coupled and thus optimized simultaneously within a set of shared weights. However, our observations reveal a previously unseen and severe bit-width interference phenomenon among highly coupled weights during optimization, leading to considerable performance degradation under a high compression ratio. To tackle this problem, we first design a bit-width scheduler to dynamically freeze the most turbulent bit-width of layers during training, to ensure the rest bit-widths converged properly. Then, taking inspiration from information theory, we present an information distortion mitigation technique to align the behaviour of the bad-performing bit-widths to the well-performing ones.

翻译：量化对于压缩过参数化深度神经网络模型并部署于资源受限设备具有重要意义。固定精度量化因数值表示能力有限而面临性能下降问题。相反，混合精度量化（MPQ）通过为各层分配异构位宽来有效压缩模型，其典型实现为搜索-重训练两阶段流程。现有工作仅关注高效确定第一阶段的最优位宽配置，却忽略了第二阶段中可观的时间成本。然而，重训练在尖端GPU上始终消耗数百GPU小时，严重阻碍部署效率。本文提出一种面向混合精度模型压缩的单次训练-搜索范式：在第一阶段，所有潜在位宽配置通过共享权重耦合，并同时优化。但观察发现，高度耦合权重在优化过程中存在先前未被发现的严重位宽干扰现象，导致高压缩比下性能显著退化。为解决该问题，我们首先设计位宽调度器，在训练中动态冻结最不稳定的层位宽，确保剩余位宽正确收敛；继而受信息论启发，提出信息失真缓解技术，将表现不佳的位宽行为对齐至表现良好的位宽。