Despite remarkable progress, existing multimodal large language models (MLLMs) are still inferior in granular visual recognition. Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming. Based on this observation, we propose a novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation (MRA). In particular, MRA adopts two visual pathways for images with different resolutions, where high-resolution visual information is embedded into the low-resolution pathway via the novel mixture-of-resolution adapters (MR-Adapters). This design also greatly reduces the input sequence length of MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL) tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g., +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR remain efficient with MRA, e.g., 20 training hours and 3$\times$ inference speed than LLaVA-1.5. Source codes are released at: https://github.com/luogen1996/LLaVA-HR.

翻译：尽管已取得显著进展，现有的大语言模型在处理细粒度视觉识别任务时仍存在不足。与以往研究不同，我们从图像分辨率的角度研究该问题，并揭示出结合低分辨率与高分辨率视觉特征可有效缓解这一缺陷。基于此发现，我们提出一种新颖高效的多模态大语言模型方法——分辨率混合自适应(Mixture-of-Resolution Adaptation, MRA)。具体而言，MRA为不同分辨率的图像构建两条视觉通路，并通过创新的分辨率混合适配器(MR-Adapters)将高分辨率视觉信息嵌入低分辨率通路。该设计还大幅缩短了多模态大语言模型的输入序列长度。为验证MRA有效性，我们将其应用于最新模型LLaVA，并命名为LLaVA-HR。在11项视觉-语言任务上的大量实验表明，LLaVA-HR在8项视觉-语言任务中超越现有模型，例如在TextVQA中提升9.4%。更重要的是，得益于MRA机制，LLaVA-HR的训练与推理始终保持高效，训练仅需20小时，推理速度较LLaVA-1.5提升3倍。源代码已开源至：https://github.com/luogen1996/LLaVA-HR。