Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their audio-visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI-Bench, a cognitively inspired benchmark that evaluates Omni-MLLMs across three stages, perception, understanding, and reasoning, through cross-modal tasks requiring joint audio-visual interpretation. This design enables fine-grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI-Bench-PriSe, an extension that probes models' primitive audio-visual sensation using unfamiliar, low-semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open-source and closed-source models reveal substantial limitations in current Omni-MLLMs. Based on these findings, we present a four-level AVI taxonomy. Overall, AVI-Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI. Project website: https://fudancvl.github.io/AVI-Bench/
翻译:近期全模态大语言模型(Omni-MLLMs)的进展实现了视觉、音频与语言的深度整合,然而由于缺乏系统性及综合性基准测试,其视听智能(AVI)仍未被充分评估。我们提出AVI-Bench——一项受认知科学启发的基准测试,通过需要联合视听解读的跨模态任务,从感知、理解与推理三个阶段对Omni-MLLMs进行评估。该设计可实现对模型能力与失效模式的细粒度诊断。为进一步评估模型在熟悉领域之外的鲁棒性,我们提出扩展版本AVI-Bench-PriSe,利用陌生且低语义的刺激探测模型的原始视听感知能力,检验其超越常见训练分布的泛化性能。针对开源与闭源模型的广泛实验揭示了当前Omni-MLLMs的显著局限性。基于发现,我们提出四层级AVI分类体系。总体而言,AVI-Bench为开发更鲁棒且可泛化的视听智能提供了原则性评估框架。项目网站:https://fudancvl.github.io/AVI-Bench/