Large language models (LLMs) have grown beyond the memory capacity of single GPU devices, necessitating quantization techniques for practical deployment. While NF4 (4-bit NormalFloat) quantization enables 4$\times$ memory reduction, inference on current NVIDIA GPUs (e.g., Ampere A100) requires expensive dequantization back to FP16 format, creating a critical performance bottleneck. This paper presents a lightweight shared memory optimization that addresses this gap through principled memory hierarchy exploitation while maintaining full ecosystem compatibility. We compare our technique against the open-source BitsAndBytes implementation, achieving 2.0--2.2$\times$ kernel speedup across three models (Gemma 27B, Qwen3 32B, and Llama3.3 70B) and up to 1.54$\times$ end-to-end improvement by leveraging the 12--15$\times$ latency advantage of shared memory over global memory access. Our optimization reduces instruction counts through simplified indexing logic while using only 64 bytes of shared memory per thread block, demonstrating that lightweight optimizations can deliver substantial performance gains with minimal engineering effort. This work provides a plug-and-play solution for the HuggingFace ecosystem that democratizes access to advanced models on existing GPU infrastructure.

翻译：大型语言模型（LLMs）的规模已超出单GPU设备的内存容量，亟需采用量化技术实现实际部署。尽管NF4（4位NormalFloat）量化可带来4倍内存缩减，但在当前NVIDIA GPU（如Ampere A100）上进行推理时，仍需将数据昂贵地反量化为FP16格式，从而形成关键性能瓶颈。本文提出一种轻量级共享内存优化方案，通过原则性的内存层次结构利用来弥合这一差距，同时保持完整的生态系统兼容性。我们将该技术开源BitsAndBytes实现进行对比，在三个模型（Gemma 27B、Qwen3 32B和Llama3.3 70B）上实现2.0--2.2倍的内核加速，以及高达1.54倍的端到端性能提升——这得益于共享内存较全局内存访问12--15倍的延迟优势。该优化通过简化索引逻辑减少指令计数，且每个线程块仅使用64字节共享内存，证明了轻量级优化能以最小工程投入实现显著性能增益。本工作为HuggingFace生态系统提供即插即用解决方案，使现有GPU基础设施上的先进模型更易普及。