Omnimodal large language models have made significant strides in unifying audio and visual modalities; however, they often face challenges in fine-grained cross-modal understanding and have difficulty with multimodal alignment. To address these limitations, we introduce OmniAgent, to our best knowledge, the first fully active perception agent that dynamically orchestrates specialized unimodal tools to achieve more fine-grained omnimodal reasoning. Unlike previous works that rely on rigid, static workflows and dense frame-captioning, we demonstrate a paradigm shift from passive response generation to active multimodal inquiry. OmniAgent employs dynamic planning to autonomously orchestrate tool invocation on demand, strategically concentrating perceptual attention on task-relevant cues. Central to our approach is a novel coarse-to-fine audio-guided perception paradigm, which leverages audio cues to localize temporal events and guide subsequent reasoning. Extensive empirical evaluations on three audio-video understanding benchmarks demonstrate that OmniAgent achieves state-of-the-art performance, surpassing leading open-source and closed-source models by substantial margins of 10% - 20% accuracy without training.

翻译：全模态大语言模型在统一音频与视觉模态方面取得了显著进展，然而它们通常在细粒度跨模态理解方面面临挑战，并在多模态对齐上存在困难。为应对这些局限，我们提出了OmniAgent，据我们所知，这是首个完全主动的感知智能体，能够动态编排专用单模态工具以实现更细粒度的全模态推理。不同于以往依赖僵化静态工作流和密集帧-描述生成的研究，我们展示了一种从被动响应生成到主动多模态查询的范式转变。OmniAgent采用动态规划技术，按需自主编排工具调用，策略性地将感知注意力集中于任务相关线索。我们方法的核心是一种新颖的由粗到细的音频引导感知范式，该范式利用音频线索定位时序事件并引导后续推理。在三个音视频理解基准上的大量实证评估表明，OmniAgent取得了最先进的性能，在不进行训练的情况下，以10%-20%的准确率优势显著超越了领先的开源与闭源模型。