The complicated architecture and high training cost of vision transformers urge the exploration of post-training quantization. However, the heavy-tailed distribution of vision transformer activations hinders the effectiveness of previous post-training quantization methods, even with advanced quantizer designs. Instead of tuning the quantizer to better fit the complicated activation distribution, this paper proposes NoisyQuant, a quantizer-agnostic enhancement for the post-training activation quantization performance of vision transformers. We make a surprising theoretical discovery that for a given quantizer, adding a fixed Uniform noisy bias to the values being quantized can significantly reduce the quantization error under provable conditions. Building on the theoretical insight, NoisyQuant achieves the first success on actively altering the heavy-tailed activation distribution with additive noisy bias to fit a given quantizer. Extensive experiments show NoisyQuant largely improves the post-training quantization performance of vision transformer with minimal computation overhead. For instance, on linear uniform 6-bit activation quantization, NoisyQuant improves SOTA top-1 accuracy on ImageNet by up to 1.7%, 1.1% and 0.5% for ViT, DeiT, and Swin Transformer respectively, achieving on-par or even higher performance than previous nonlinear, mixed-precision quantization.

翻译：视觉Transformer复杂的架构与高昂的训练成本推动了训练后量化的研究。然而，视觉Transformer激活值的重尾分布阻碍了现有训练后量化方法（即使采用高级量化器设计）的有效性。本文并未通过调整量化器来适配复杂的激活分布，而是提出NoisyQuant——一种与量化器无关的增强方法，用于提升视觉Transformer的训练后激活量化性能。我们获得了一个出人意料的理论发现：对于给定量化器，向被量化值添加固定的均匀噪声偏置，可在可证明条件下显著降低量化误差。基于这一理论洞见，NoisyQuant首次成功通过添加噪声偏置主动改变重尾激活分布，以适配给定量化器。大量实验表明，NoisyQuant能以极小的计算开销大幅提升视觉Transformer的训练后量化性能。例如，在线性均匀6比特激活量化中，NoisyQuant使ViT、DeiT和Swin Transformer在ImageNet上的SOTA top-1准确率分别提升高达1.7%、1.1%和0.5%，达到甚至超越先前非线性混合精度量化的性能。