The rapidly developing Large Vision Language Models (LVLMs) have shown notable capabilities on a range of multi-modal tasks, but still face the hallucination phenomena where the generated texts do not align with the given contexts, significantly restricting the usages of LVLMs. Most previous work detects and mitigates hallucination at the coarse-grained level or requires expensive annotation (e.g., labeling by proprietary models or human experts). To address these issues, we propose detecting and mitigating hallucinations in LVLMs via fine-grained AI feedback. The basic idea is that we generate a small-size sentence-level hallucination annotation dataset by proprietary models, whereby we train a hallucination detection model which can perform sentence-level hallucination detection, covering primary hallucination types (i.e., object, attribute, and relationship). Then, we propose a detect-then-rewrite pipeline to automatically construct preference dataset for training hallucination mitigating model. Furthermore, we propose differentiating the severity of hallucinations, and introducing a Hallucination Severity-Aware Direct Preference Optimization (HSA-DPO) for mitigating hallucination in LVLMs by incorporating the severity of hallucinations into preference learning. Extensive experiments demonstrate the effectiveness of our method.

翻译：快速发展的多模态大语言模型在多种多模态任务中展现出显著能力，但依然存在生成文本与给定上下文不一致的幻觉现象，这严重限制了模型的应用范围。现有方法多采用粗粒度检测或依赖昂贵标注（如通过专有模型或人类专家标注）。为解决上述问题，我们提出通过细粒度AI反馈实现视觉语言模型幻觉检测与缓解。核心思路为：首先利用专有模型生成小规模句子级幻觉标注数据集，基于该数据训练可执行句子级幻觉检测的检测模型，覆盖主要幻觉类型（物体、属性、关系）。继而构建"检测-重写"流水线来自动生成偏好数据集，用于训练幻觉缓解模型。进一步，我们提出区分幻觉严重程度，并引入幻觉严重程度感知直接偏好优化方法，通过将幻觉严重度融入偏好学习来缓解视觉语言模型的幻觉问题。大量实验证明了该方法的有效性。