We introduce a quantization-aware training algorithm that guarantees avoiding numerical overflow when reducing the precision of accumulators during inference. We leverage weight normalization as a means of constraining parameters during training using accumulator bit width bounds that we derive. We evaluate our algorithm across multiple quantized models that we train for different tasks, showing that our approach can reduce the precision of accumulators while maintaining model accuracy with respect to a floating-point baseline. We then show that this reduction translates to increased design efficiency for custom FPGA-based accelerators. Finally, we show that our algorithm not only constrains weights to fit into an accumulator of user-defined bit width, but also increases the sparsity and compressibility of the resulting weights. Across all of our benchmark models trained with 8-bit weights and activations, we observe that constraining the hidden layers of quantized neural networks to fit into 16-bit accumulators yields an average 98.2% sparsity with an estimated compression rate of 46.5x all while maintaining 99.2% of the floating-point performance.

翻译：我们提出了一种量化感知训练算法，可在推理过程中降低累加器精度时保证避免数值溢出。我们利用权重归一化作为训练中约束参数的手段，并推导了累加器位宽边界。我们在针对不同任务训练的多个量化模型上评估了该算法，结果表明我们的方法能在保持模型相对于浮点基线精度的同时降低累加器精度。接着，我们证明这种精度的降低可转化为基于FPGA定制加速器设计效率的提升。最后，我们指出该算法不仅将权重约束在用户指定位宽的累加器内，还增强了所得权重的稀疏性和可压缩性。在所有采用8位权重和激活值训练的基准模型中，我们发现将量化神经网络隐藏层约束在16位累加器内可实现平均98.2%的稀疏度，预估压缩率达46.5倍，同时保持浮点性能的99.2%。