Large language models(LLMs) exhibit excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models.

翻译：大语言模型（LLMs）在各类任务中展现出卓越性能，但随之带来了显著的计算与存储成本。对这些模型进行量化是缓解该问题的有效途径。然而，现有方法难以在模型精度与硬件效率之间取得平衡。为此，我们提出AWEQ——一种无需额外训练开销的后训练方法。该方法在超低位量化以及8位权重与激活（W8A8）量化场景中均表现优异。我们观察到，权重量化的难度低于激活量化。AWEQ通过通道均衡技术将激活量化的难点转移至权重，实现两者量化难度的平衡，从而最大化模型性能。我们进一步优化了均衡方法以减小量化偏置误差，确保模型的鲁棒性。在LLaMA、OPT等主流模型上的大量实验表明，AWEQ在所有现有大模型后训练量化方法中性能最优。