For extreme low-bit quantization of large language models (LLMs), Double Binary Factorization (DBF) is attractive as it enables efficient inference without sacrificing accuracy. However, the scaling parameters of DBF are too restrictive; after factoring out signs, all rank components share the same magnitude profile, resulting in performance saturation. We propose Multi-envelope DBF (MDBF), which retains a shared pair of 1-bit sign bases but replaces the single envelope with a rank-$l$ envelope. By sharing sign matrices among envelope components, MDBF effectively maintains a binary carrier and utilizes the limited memory budget for magnitude expressiveness. We also introduce a closed-form initialization and an alternating refinement method to optimize MDBF. Across the LLaMA and Qwen families, MDBF enhances perplexity and zero-shot accuracy over previous binary formats at matched bits per weight while preserving the same deployment-friendly inference primitive.

翻译：对于大型语言模型（LLMs）的极端低位量化，双二进制分解（DBF）因其能在不牺牲精度的情况下实现高效推理而备受关注。然而，DBF的缩放参数限制过强；在分解出符号后，所有秩分量共享相同的幅度分布，导致性能饱和。我们提出多包络DBF（MDBF），该方法保留共享的一对1位符号基，但将单一包络替换为秩-$l$包络。通过在包络分量间共享符号矩阵，MDBF有效地保持了二进制载体，并将有限的内存预算用于提升幅度表达能力。我们还提出了一种闭式初始化方法和一种交替优化方法来训练MDBF。在LLaMA和Qwen系列模型上的实验表明，在保持相同部署友好型推理原语且每权重比特数匹配的条件下，MDBF相比以往的二进制格式，在困惑度和零样本准确率上均有提升。