Post training quantization is essential for deploying large language models (LLMs) on resource constrained hardware, yet state of the art methods enforce uniform bit widths across layers, yielding suboptimal accuracy efficiency trade offs. We present RAMP (Reinforcement Adaptive Mixed Precision), an off policy Soft Actor Critic framework that learns per layer bit width assignments to minimize perplexity under a global bit budget. The policy conditions on an 11 dimensional embedding of activation statistics, weight properties, and structural descriptors, enabling zero shot transfer across model families and scales. To enable stable sub 4 bit quantization, we introduce Scale Folding, a preconditioning technique that migrates activation outliers into weights via per channel scaling and normalization layer compensation. A quality prioritized reward with asymmetric penalties and budget cliffs drives rapid convergence. On Llama 2 7B, RAMP achieves 5.54 perplexity at 3.68GB (3.65 effective bits), outperforming uniform 4 bit AWQ (5.60 at 3.90 GB) and GPTQ by 6% in size and 1% to3% in quality. Critically, a policy trained only on Llama 2 7B generalizes zero shot to Llama 2 13B and Mistral 7B, often surpassing target specific training, supporting the hypothesis that quantization sensitivity is primarily architectural. The HALO pipeline exports allocations to GGUF format for kernel free inference on CPUs, GPUs, and edge devices, retaining 99.5% of FP16 commonsense reasoning performance.

翻译：后训练量化对于在资源受限硬件上部署大语言模型至关重要，但现有先进方法在层间强制采用统一位宽，导致精度与效率的权衡次优。我们提出RAMP（强化自适应混合精度方法），这是一种离策略软演员-评论家框架，通过学习逐层位宽分配在全局位预算下最小化困惑度。该策略以激活统计量、权重属性及结构描述符的11维嵌入为条件，支持跨模型族与尺度的零样本迁移。为实现稳定的亚4位量化，我们引入尺度折叠预处理技术，通过逐通道缩放与归一化层补偿将激活异常值迁移至权重中。采用质量优先奖励函数，结合非对称惩罚与预算悬崖机制，驱动快速收敛。在Llama 2 7B上，RAMP以3.68GB（3.65有效位）实现5.54困惑度，较均匀4位AWQ（3.90GB时5.60困惑度）和GPTQ在体积上提升6%，质量上提升1%至3%。关键的是，仅在Llama 2 7B上训练的策略可零样本泛化至Llama 2 13B与Mistral 7B，且常优于目标专用训练，支持了量化敏感性主要源于架构的假设。HALO流水线将分配导出为GGUF格式，支持CPU、GPU及边缘设备上的无内核推理，保留FP16常识推理性能的99.5%。