Quantization-aware training (QAT) simulates a quantization process during training to lower bit-precision of weights/activations. It learns quantized weights indirectly by updating latent weights,i.e., full-precision inputs to a quantizer, using gradient-based optimizers. We claim that coupling a user-defined learning rate (LR) with these optimizers is sub-optimal for QAT. Quantized weights transit discrete levels of a quantizer, only if corresponding latent weights pass transition points, where the quantizer changes discrete states. This suggests that the changes of quantized weights are affected by both the LR for latent weights and their distributions. It is thus difficult to control the degree of changes for quantized weights by scheduling the LR manually. We conjecture that the degree of parameter changes in QAT is related to the number of quantized weights transiting discrete levels. Based on this, we introduce a transition rate (TR) scheduling technique that controls the number of transitions of quantized weights explicitly. Instead of scheduling a LR for latent weights, we schedule a target TR of quantized weights, and update the latent weights with a novel transition-adaptive LR (TALR), enabling considering the degree of changes for the quantized weights during QAT. Experimental results demonstrate the effectiveness of our approach on standard benchmarks.

翻译：量化感知训练（QAT）通过在训练过程中模拟量化操作来降低权重/激活值的比特精度。它通过基于梯度的优化器更新潜权重（即量化器的全精度输入）来间接学习量化权重。我们认为，在QAT中将用户定义的学习率（LR）与这些优化器耦合是次优的。量化权重仅在对应的潜权重穿越量化器状态发生突变的转移点时，才会跨越量化器的离散层级。这表明量化权重的变化同时受到潜权重学习率及其分布的影响。因此，通过手动调度学习率难以控制量化权重的变化程度。我们推测QAT中参数变化程度与跨越离散层级的量化权重数量相关。基于此，我们提出一种转移率（TR）调度技术，显式控制量化权重的转移次数。该方法不再为潜权重调度学习率，而是为量化权重调度目标转移率，并采用新型转移自适应学习率（TALR）更新潜权重，从而在QAT过程中兼顾量化权重的变化程度。实验结果表明，我们的方法在标准基准测试中具有显著有效性。