We present the All-Seeing Project V2: a new model and dataset designed for understanding object relations in images. Specifically, we propose the All-Seeing Model V2 (ASMv2) that integrates the formulation of text generation, object localization, and relation comprehension into a relation conversation (ReC) task. Leveraging this unified task, our model excels not only in perceiving and recognizing all objects within the image but also in grasping the intricate relation graph between them, diminishing the relation hallucination often encountered by Multi-modal Large Language Models (MLLMs). To facilitate training and evaluation of MLLMs in relation understanding, we created the first high-quality ReC dataset ({AS-V2) which is aligned with the format of standard instruction tuning data. In addition, we design a new benchmark, termed Circular-based Relation Probing Evaluation (CRPE) for comprehensively evaluating the relation comprehension capabilities of MLLMs. Notably, our ASMv2 achieves an overall accuracy of 52.04 on this relation-aware benchmark, surpassing the 43.14 of LLaVA-1.5 by a large margin. We hope that our work can inspire more future research and contribute to the evolution towards artificial general intelligence. Our project is released at https://github.com/OpenGVLab/all-seeing.

翻译：我们提出全视项目V2——一个用于理解图像中对象关系的新模型与数据集。具体而言，我们提出全视模型V2（ASMv2），该模型将文本生成、目标定位与关系理解整合为关系对话（ReC）任务。借助这一统一任务，我们的模型不仅能感知并识别图像中的所有对象，还能掌握对象间的复杂关系图，从而减轻多模态大语言模型（MLLMs）常见的关系幻觉现象。为促进MLLMs在关系理解方面的训练与评估，我们创建了首个与标准指令微调数据格式对齐的高质量ReC数据集（AS-V2）。此外，我们设计了一个名为圆形基关系探测评估（CRPE）的新型基准，用于全面评估MLLMs的关系理解能力。值得注意的是，我们的ASMv2在此关系感知基准上达到52.04的整体准确率，大幅超越LLaVA-1.5的43.14。我们希望这项工作能激发更多未来研究，并为迈向人工通用智能的演进做出贡献。我们的项目已发布于https://github.com/OpenGVLab/all-seeing。