Large language models (LLMs) have revolutionized natural language processing tasks. However, their practical deployment is hindered by their immense memory and computation requirements. Although recent post-training quantization (PTQ) methods are effective in reducing memory footprint and improving the computational efficiency of LLM, they hand-craft quantization parameters, leading to low performance, especially in extremely low-bit quantization. To tackle this issue, we introduce an Omnidirectionally calibrated Quantization (\textbf{OmniQuant}) technique for LLMs, which achieves good performance in diverse quantization settings while maintaining the computational efficiency of PTQ by efficiently optimizing various quantization parameters. OmniQuant comprises two innovative components including Learnable Weight Clipping (LWC) and Learnable Equivalent Transformation (LET). LWC modulates the extreme values of weights by optimizing the clipping threshold. Meanwhile, LET tackles activation outliers by shifting the challenge of quantization from activations to weights. Operating within a differentiable framework using block-wise error minimization, OmniQuant can optimize the quantization process efficiently for both weight-only and weight-activation quantization. For instance, the LLaMA-2 model family size 7-70B can be processed with OmniQuant on a single A100-40G GPU within 1-16 hours using 128 samples. Extensive experiments validate OmniQuant's superior performance across diverse quantization configurations such as W4A4 (4-bit weight, 4-bit activation), W6A6, W4A16, W3A16, and W2A16. Additionally, OmniQuant demonstrates effectiveness in instruction-tuned models and delivers notable improvements in inference speed and memory reduction on real devices. Codes are available at \url{https://github.com/OpenGVLab/OmniQuant}.

翻译：大型语言模型（LLMs）革新了自然语言处理任务。然而，其巨大内存与计算需求阻碍了实际部署。尽管近期提出的训练后量化（PTQ）方法在减少LLM内存占用和提升计算效率方面效果显著，但这类方法依赖手工设计的量化参数，导致性能低下，尤其在极低位宽量化场景中。为解决该问题，我们提出面向LLM的全方位校准量化技术（OmniQuant），该方法在保持PTQ计算效率的同时，通过高效优化多种量化参数，在不同量化配置下均取得优异性能。OmniQuant包含两项创新组件：可学习权重裁剪（LWC）与可学习等效变换（LET）。LWC通过优化裁剪阈值调节权重的极值分布，而LET通过将量化困难从激活值转移至权重来处理激活异常值。基于逐块误差最小化的可微分框架，OmniQuant能够高效优化权重量化与权重-激活联合量化流程。以LLaMA-2系列模型（7B-70B）为例，使用128个样本在单张A100-40G GPU上即可在1-16小时内完成处理。大量实验表明，OmniQuant在W4A4（4比特权重，4比特激活）、W6A6、W4A16、W3A16及W2A16等多种量化配置中均展现出卓越性能。此外，OmniQuant在指令微调模型上同样表现有效，并在实际设备上显著提升推理速度与内存压缩效果。相关代码已开源至\url{https://github.com/OpenGVLab/OmniQuant}。