Post-training quantization at the 2-bit level enables low-cost deployment and inference acceleration for large language models (LLMs). Scalar quantization (SQ) and vector quantization (VQ) are two primary quantization methods, however, the former suffers from significant performance degradation, and the latter incurs computational and storage overhead. We propose UniSVQ, a unified 2-bit quantization framework that bridges scalar and vector quantization by parameterizing codewords as an affine transform of integer lattices. This structure preserves compatibility with optimized integer kernels while retaining much of VQ's flexibility. We further introduce a data-driven block-wise fine-tuning strategy to directly minimize quantization reconstruction error. Extensive experiments across multiple LLM families and zero-shot benchmarks demonstrate that UniSVQ consistently outperforms state-of-the-art SQ methods and achieves performance comparable to advanced VQ methods, while providing higher inference throughput.

翻译：后训练量化在2比特级别上能够实现大语言模型（LLMs）的低成本部署与推理加速。标量量化（SQ）和向量量化（VQ）是两种主要的量化方法，然而前者存在显著的性能下降，后者则带来计算和存储开销。我们提出UniSVQ，一个统一的2比特量化框架，通过将码字参数化为整数格点的仿射变换，桥接了标量量化与向量量化。该结构在保留VQ大部分灵活性的同时，保持了与优化整数内核的兼容性。我们进一步引入数据驱动的逐块微调策略，以直接最小化量化重构误差。在多个LLM系列和零样本基准上的大量实验表明，UniSVQ持续优于最先进的SQ方法，性能与先进的VQ方法相当，同时提供更高的推理吞吐量。