Large language models (LLMs) deliver strong performance, but their high compute and memory costs make deployment difficult in resource-constrained scenarios. Weight-only post-training quantization (PTQ) is appealing, as it reduces memory usage and enables practical speedup without low-bit operators or specialized hardware. However, accuracy often degrades significantly in weight-only PTQ at sub-4-bit precision, and our analysis identifies two main causes: (1) down-projection matrices are a well-known quantization bottleneck, but maintaining their fidelity often requires extra bit-width; (2) weight quantization induces activation deviations, but effective correction strategies remain underexplored. To address these issues, we propose D$^2$Quant, a novel weight-only PTQ framework that improves quantization from both the weight and activation perspectives. On the weight side, we design a Dual-Scale Quantizer (DSQ) tailored to down-projection matrices, with an absorbable scaling factor that significantly improves accuracy without increasing the bit budget. On the activation side, we propose Deviation-Aware Correction (DAC), which incorporates a mean-shift correction within LayerNorm to mitigate quantization-induced activation distribution shifts. Extensive experiments across multiple LLM families and evaluation metrics show that D$^2$Quant delivers superior performance for weight-only PTQ at sub-4-bit precision. The code and models will be available at https://github.com/XIANGLONGYAN/D2Quant.

翻译：大语言模型（LLM）展现出强大的性能，但其高昂的计算与内存成本使其在资源受限场景下的部署面临困难。仅权重的训练后量化（PTQ）因其能降低内存占用，并在无需低位宽算子或专用硬件的条件下实现实际加速而备受关注。然而，在低于4位宽的精度下，仅权重的PTQ通常会导致模型精度显著下降。我们的分析识别出两个主要原因：（1）下投影矩阵是众所周知的量化瓶颈，但保持其保真度通常需要额外的位宽；（2）权重量化会引发激活值偏差，而有效的校正策略仍未得到充分探索。为解决这些问题，我们提出了D$^2$Quant，一个新颖的仅权重PTQ框架，从权重和激活两个角度改进量化。在权重方面，我们设计了一种专为下投影矩阵定制的双尺度量化器（DSQ），其包含一个可吸收的缩放因子，能在不增加位宽预算的情况下显著提升精度。在激活方面，我们提出了偏差感知校正（DAC），该方法在LayerNorm中引入均值偏移校正，以减轻量化引起的激活分布偏移。在多个LLM家族和评估指标上进行的大量实验表明，D$^2$Quant在低于4位宽的精度下，为仅权重的PTQ提供了卓越的性能。代码与模型将在 https://github.com/XIANGLONGYAN/D2Quant 发布。