Modern applications commonly leverage large, multi-modal foundation models. These applications often feature complex workflows that demand the storage and usage of similar models in multiple precisions. A straightforward approach is to maintain a separate file for each model precision (e.g., INT8, BF16), which is indeed the approach taken by many model providers such as HuggingFace and Ollama. However, this approach incurs excessive storage costs since a higher precision model (e.g., BF16) is a strict superset of a lower precision model (e.g., INT8) in terms of information. Unfortunately, simply maintaining only the higher-precision model and requiring every user to dynamically convert the model precision is not desirable because every user of lower precision models must pay the cost for model download and precision conversion. In this paper, we present QStore, a unified, lossless compression format for simultaneously storing a model in two (high and low) precisions efficiently. Instead of storing low-precision and high-precision models separately, QStore stores low-precision model and only the residual information needed to reconstruct high-precision models. The size of residual information is significantly smaller than the original high-precision models, thus achieving high savings in storage cost. Moreover, QStore does not compromise the speed of model loading. The low-precision models can be loaded quickly just like before. The high-precision models can also be reconstructed efficiently in memory by merging low-precision data and the residual with QStore's lightweight decoding logic. We evaluate QStore for compressing multiple precisions of popular foundation models, and show that QStore reduces overall storage footprint by up to 2.2x (45% of the original size) while enabling up to 1.7x and 1.8x faster model saving and loading versus existing approaches.

翻译：现代应用通常利用大型多模态基础模型。这些应用通常具有复杂的工作流程，需要以多种精度存储和使用相似的模型。一种直接的方法是为每种模型精度（例如INT8、BF16）维护单独的文件，这确实是许多模型提供商（如HuggingFace和Ollama）采用的方法。然而，这种方法会产生过高的存储成本，因为从信息角度看，较高精度的模型（例如BF16）是较低精度模型（例如INT8）的严格超集。不幸的是，仅维护较高精度模型并要求每个用户动态转换模型精度并不可取，因为每个使用较低精度模型的用户都必须承担模型下载和精度转换的开销。本文提出QStore，一种统一的无损压缩格式，用于高效地同时存储两种（高和低）精度的模型。QStore不单独存储低精度和高精度模型，而是存储低精度模型以及重建高精度模型所需的残差信息。残差信息的大小显著小于原始高精度模型，从而实现了存储成本的大幅节省。此外，QStore不损害模型加载速度。低精度模型可以像以前一样快速加载。高精度模型也可以通过合并低精度数据和残差，并利用QStore的轻量级解码逻辑在内存中高效重建。我们评估了QStore在压缩流行基础模型的多种精度方面的表现，结果表明，与现有方法相比，QStore将总体存储占用减少了高达2.2倍（原始大小的45%），同时使模型保存和加载速度分别提升了高达1.7倍和1.8倍。