Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs. However, existing PTQ methods only focus on handling the outliers within one layer or one block, which ignores the dependency of blocks and leads to severe performance degradation in low-bit settings. In this paper, we propose CBQ, a cross-block reconstruction-based PTQ method for LLMs. CBQ employs a cross-block dependency using a homologous reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation. Furthermore, CBQ incorporates a coarse-to-fine preprocessing (CFP) strategy for suppressing weight and activation outliers, coupled with an adaptive LoRA-Rounding technique for precise weight quantization. These innovations enable CBQ to not only handle extreme outliers effectively but also improve overall quantization accuracy. Extensive experiments show that CBQ achieves superior low-bit quantization (W4A4, W4A8, W2A16) and outperforms existing state-of-the-art methods across various LLMs and datasets. Notably, CBQ quantizes the 4-bit LLAMA1-65B model within only 4.3 hours on a single GPU, achieving a commendable tradeoff between performance and quantization efficiency.

翻译：后训练量化（PTQ）在极低开销压缩大语言模型（LLMs）中发挥了关键作用。然而，现有PTQ方法仅关注处理单层或单块内的异常值，忽视了块间的依赖关系，导致低位设置下性能严重下降。本文提出CBQ——一种基于跨块重构的LLMs后训练量化方法。CBQ采用同源重构方案建立跨块依赖性，通过构建跨多个块的长程依赖关系来最小化误差累积。此外，CBQ整合了从粗到细的预处理（CFP）策略以抑制权重和激活异常值，并结合自适应LoRA-Rounding技术实现精确权重量化。这些创新使CBQ不仅能有效处理极端异常值，还能提升整体量化精度。大量实验表明，CBQ在低位量化（W4A4、W4A8、W2A16）中表现优异，并在多种LLMs及数据集上超越现有最先进方法。值得注意的是，CBQ在单GPU上仅需4.3小时即可完成4位LLAMA1-65B模型的量化，实现了性能与量化效率的优异平衡。