Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, e.g., story writing, where the models are expected to generate novel content beyond the given image. In these tasks, it is inappropriate to simply regard such imagined novel content as hallucinations. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE)-a novel method to assess LVLMs' hallucinations in voluntary imagination tasks via presence evaluation. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. The consistency between the model's interpretation and the object's presence in the image is then used to determine whether the model hallucinates when generating the response. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.

翻译：当前针对大型视觉语言模型（LVLMs）幻觉现象的研究大多集中于禁止输出图像中不存在内容的客观描述任务。然而，对于自愿想象任务（例如故事创作）中的幻觉问题关注甚少，这类任务要求模型基于给定图像生成超越图像内容的新颖信息。在此类任务中，将模型生成的想象内容简单归类为幻觉并不恰当。为突破这一局限，我们提出自愿想象对象存在性评估（VOPE）——一种通过存在性评估来衡量LVLMs在自愿想象任务中幻觉现象的新方法。具体而言，VOPE通过设置复核式问题，评估LVLM如何解释其自身响应中想象对象的存在性。通过对比模型解释与图像中对象实际存在性的一致性，可判定模型在生成响应时是否产生幻觉。我们将VOPE应用于多个主流LVLMs及幻觉缓解方法，揭示出两个关键发现：（1）多数LVLMs在自愿想象过程中存在严重幻觉，且对想象对象的存在性评估表现显著不佳；（2）现有幻觉缓解方法在自愿想象任务中效果有限，这将成为未来研究的重要方向。