Post-training quantization (PTQ) attracts increasing attention due to its convenience in deploying quantized neural networks. Rounding is the primary source of quantization error, for which previous works adopt the rounding-to-nearest scheme with a constant border of 0.5. This work demonstrates that optimizing rounding schemes can improve model accuracy. By replacing the constant border with a simple border function, we can obtain the minimal error for multiplying two numbers and eliminate the bias of its expected value, which further benefits model accuracy. Based on this insight, we approximate the border function to make the incurred overhead negligible. We also jointly optimize propagated errors and global errors. We finally propose our AQuant framework, which can learn the border function automatically. Extensive experiments show that AQuant achieves noticeable improvements compared with state-of-the-art works and pushes the accuracy of ResNet-18 up to 60.31% under the 2-bit weight and activation post-training quantization.

翻译：训练后量化（PTQ）因其在部署量化神经网络时的便捷性而日益受到关注。舍入是量化误差的主要来源，以往的工作采用固定边界为0.5的最近邻舍入方案。本研究表明，优化舍入方案可提升模型精度。通过将固定边界替换为简单的边界函数，我们可以实现两数相乘的最小误差，并消除其期望值的偏差，这进一步提升了模型精度。基于这一见解，我们对边界函数进行近似处理，使产生的额外开销可忽略不计。此外，我们还联合优化了传播误差和全局误差。最终提出了AQuant框架，能够自动学习边界函数。大量实验表明，与最新方法相比，AQuant取得了显著改进，并在2比特权重和激活训练后量化下将ResNet-18的精度提升至60.31%。