Post-training quantization (PTQ), which only requires a tiny dataset for calibration without end-to-end retraining, is a light and practical model compression technique. Recently, several PTQ schemes for vision transformers (ViTs) have been presented; unfortunately, they typically suffer from non-trivial accuracy degradation, especially in low-bit cases. In this paper, we propose RepQ-ViT, a novel PTQ framework for ViTs based on quantization scale reparameterization, to address the above issues. RepQ-ViT decouples the quantization and inference processes, where the former employs complex quantizers and the latter employs scale-reparameterized simplified quantizers. This ensures both accurate quantization and efficient inference, which distinguishes it from existing approaches that sacrifice quantization performance to meet the target hardware. More specifically, we focus on two components with extreme distributions: post-LayerNorm activations with severe inter-channel variation and post-Softmax activations with power-law features, and initially apply channel-wise quantization and log$\sqrt{2}$ quantization, respectively. Then, we reparameterize the scales to hardware-friendly layer-wise quantization and log2 quantization for inference, with only slight accuracy or computational costs. Extensive experiments are conducted on multiple vision tasks with different model variants, proving that RepQ-ViT, without hyperparameters and expensive reconstruction procedures, can outperform existing strong baselines and encouragingly improve the accuracy of 4-bit PTQ of ViTs to a usable level. Code is available at https://github.com/zkkli/RepQ-ViT.

翻译：训练后量化（PTQ）仅需极少量数据进行校准，无需端到端重新训练，是一种轻量且实用的模型压缩技术。近期虽有多种针对视觉Transformer（ViT）的PTQ方案被提出，但它们在低比特场景下仍存在显著的精度损失。本文提出RepQ-ViT——一种基于量化尺度重参数化的新型ViT PTQ框架以解决上述问题。RepQ-ViT将量化过程与推理过程解耦：前者采用复杂量化器，后者则使用尺度重参数化后的简化量化器。这既保证了量化精度，又实现了高效推理，与现有为适配目标硬件而牺牲量化性能的方法形成鲜明对比。具体而言，我们聚焦两类具有极端分布特征的组件：存在严重跨通道变化的LayerNorm后激活值与呈现幂律特征的Softmax后激活值，并分别采用通道量化和log√2量化。随后通过尺度重参数化，将这些量化方式转化为硬件友好的逐层量化和log2量化用于推理，仅引入微小的精度或计算开销。我们在多种视觉任务及不同模型变体上进行大量实验，证明RepQ-ViT无需超参数调优和复杂重建流程，即可超越现有强基线方法，并令人振奋地将4比特ViT PTQ的精度提升至可用水平。代码已开源至https://github.com/zkkli/RepQ-ViT。