By combining natural language understanding, generation capabilities, and breadth of knowledge of large language models with image perception, recent large vision language models (LVLMs) have shown unprecedented visual reasoning capabilities. However, the generated text often suffers from inaccurate grounding in the visual input, resulting in errors such as hallucination of nonexistent scene elements, missing significant parts of the scene, and inferring incorrect attributes of and relationships between objects. To address these issues, we introduce a novel framework, ViGoR(Visual Grounding Through Fine-Grained Reward Modeling) that utilizes fine-grained reward modeling to significantly enhance the visual grounding of LVLMs over pre-trained baselines. This improvement is efficiently achieved using much cheaper human evaluations instead of full supervisions, as well as automated methods. We show the effectiveness of our approach through a variety of evaluation methods and benchmarks. Additionally, we plan to release our human annotation comprising approximately 16,000 images and generated text pairs with fine-grained evaluations to contribute to related research in the community.

翻译：通过将大型语言模型在自然语言理解、生成能力及知识广度与图像感知相结合，近年来发展的大型视觉语言模型（LVLM）展现了前所未有的视觉推理能力。然而，模型生成的文本常存在对视觉输入定位不准确的问题，导致产生场景中不存在元素的幻觉、遗漏场景关键部分、推断物体属性及关系错误等现象。为解决这些问题，我们提出了一个新型框架ViGoR（基于细粒度奖励建模的视觉定位），该框架通过细粒度奖励建模，在预训练基线基础上显著增强了LVLM的视觉定位能力。这种提升可通过远低于完全监督的代价高效实现——借助人类评估（成本更低）及自动化方法。我们通过多种评估方法与基准实验验证了该方法的有效性。此外，计划公开包含约16,000对图像-生成文本对及细粒度评估的人工标注数据集，以推动相关领域研究。