Multimodal large language models (MLLMs) typically rely on a single late-layer feature from a frozen vision encoder, leaving the encoder's rich hierarchy of visual cues under-utilized. MLLMs still suffer from visually ungrounded hallucinations, often relying on language priors rather than image evidence. While many prior mitigation strategies operate on the text side, they leave the visual representation unchanged and do not exploit the rich hierarchy of features encoded across vision layers. Existing multi-layer fusion methods partially address this limitation but remain static, applying the same layer mixture regardless of the query. In this work, we introduce TGIF (Text-Guided Inter-layer Fusion), a lightweight module that treats encoder layers as depth-wise "experts" and predicts a prompt-dependent fusion of visual features. TGIF follows the principle of direct external fusion, requires no vision-encoder updates, and adds minimal overhead. Integrated into LLaVA-1.5-7B, TGIF provides consistent improvements across hallucination, OCR, and VQA benchmarks, while preserving or improving performance on ScienceQA, GQA, and MMBench. These results suggest that query-conditioned, hierarchy-aware fusion is an effective way to strengthen visual grounding and reduce hallucination in modern MLLMs.

翻译：多模态大语言模型（MLLMs）通常仅依赖冻结视觉编码器的单一深层特征，未能充分利用编码器所蕴含的丰富视觉层级线索。当前MLLMs仍普遍存在视觉依据不足的幻觉问题，往往依赖语言先验而非图像证据。尽管已有许多缓解策略在文本侧进行操作，但这些方法未改变视觉表征，亦未利用视觉各层级编码的丰富特征。现有的多层融合方法部分解决了这一局限，但仍属于静态融合，即无论查询内容如何均采用相同的层级混合方式。本研究提出TGIF（文本引导的层间融合），这是一个轻量级模块，将编码器各层视为深度维度的“专家”，并预测基于提示的视觉特征融合方案。TGIF遵循直接外部融合原则，无需更新视觉编码器，且仅引入极小计算开销。将TGIF集成至LLaVA-1.5-7B模型后，在幻觉检测、OCR及VQA基准测试中均取得稳定提升，同时在ScienceQA、GQA和MMBench基准上保持或改进了原有性能。这些结果表明，基于查询条件的层级感知融合是增强现代MLLMs视觉基础、减少幻觉的有效途径。