Large language models (LLMs) deliver impressive results for a variety of tasks, but state-of-the-art systems require fast GPUs with large amounts of memory. To reduce both the memory and latency of these systems, practitioners quantize their learned parameters, typically at half precision. A growing body of research focuses on preserving the model performance with more aggressive bit widths, and some work has been done to apply these strategies to other models, like vision transformers. In our study we investigate how a variety of quantization methods, including state-of-the-art GPTQ and AWQ, can be applied effectively to multimodal pipelines comprised of vision models, language models, and their connectors. We address how performance on captioning, retrieval, and question answering can be affected by bit width, quantization method, and which portion of the pipeline the quantization is used for. Results reveal that ViT and LLM exhibit comparable importance in model performance, despite significant differences in parameter size, and that lower-bit quantization of the LLM achieves high accuracy at reduced bits per weight (bpw). These findings provide practical insights for efficient deployment of MLLMs and highlight the value of exploration for understanding component sensitivities in multimodal models. Our code is available at https://github.com/gautomdas/mmq.

翻译：大型语言模型（LLM）在各种任务中展现出卓越的性能，但最先进的系统需要具备大容量内存的高速GPU。为了降低这些系统的内存需求和延迟，实践者通常将其学习参数量化为半精度。越来越多的研究致力于在更激进的比特宽度下保持模型性能，并且已有部分工作将这些策略应用于其他模型，如视觉Transformer。在本研究中，我们探讨了包括最先进的GPTQ和AWQ在内的多种量化方法，如何有效应用于由视觉模型、语言模型及其连接器构成的多模态流程。我们分析了比特宽度、量化方法以及量化应用于流程的哪个部分，如何影响图像描述、检索和问答任务的性能。结果表明，尽管参数量存在显著差异，ViT和LLM对模型性能具有相当的重要性，并且对LLM进行更低比特的量化能够在减少每权重比特数（bpw）的同时实现高精度。这些发现为高效部署多模态大语言模型提供了实用见解，并凸显了探索多模态模型中组件敏感性的价值。我们的代码发布于https://github.com/gautomdas/mmq。