Omni-modal Large Language Models (Omni-LLMs) have demonstrated strong capabilities in audio-video understanding tasks. However, their reliance on long multimodal token sequences leads to substantial computational overhead. Despite this challenge, token compression methods designed for Omni-LLMs remain limited. To bridge this gap, we propose OmniSIFT (Omni-modal Spatio-temporal Informed Fine-grained Token compression), a modality-asymmetric token compression framework tailored for Omni-LLMs. Specifically, OmniSIFT adopts a two-stage compression strategy: (i) a spatio-temporal video pruning module that removes video redundancy arising from both intra-frame structure and inter-frame overlap, and (ii) a vision-guided audio selection module that filters audio tokens. The entire framework is optimized end-to-end via a differentiable straight-through estimator. Extensive experiments on five representative benchmarks demonstrate the efficacy and robustness of OmniSIFT. Notably, for Qwen2.5-Omni-7B, OmniSIFT introduces only 4.85M parameters while maintaining lower latency than training-free baselines such as OmniZip. With merely 25% of the original token context, OmniSIFT consistently outperforms all compression baselines and even surpasses the performance of the full-token model on several tasks.

翻译：全模态大语言模型（Omni-LLMs）在音视频理解任务中展现出强大能力。然而，其对长多模态令牌序列的依赖导致了巨大的计算开销。尽管存在这一挑战，专为Omni-LLMs设计的令牌压缩方法仍然有限。为填补这一空白，我们提出了OmniSIFT（全模态时空感知细粒度令牌压缩），一种专为Omni-LLMs定制的模态非对称令牌压缩框架。具体而言，OmniSIFT采用两阶段压缩策略：（i）一个时空视频剪枝模块，用于消除由帧内结构和帧间重叠引起的视频冗余；（ii）一个视觉引导的音频选择模块，用于过滤音频令牌。整个框架通过可微分直通估计器进行端到端优化。在五个代表性基准测试上的大量实验证明了OmniSIFT的有效性和鲁棒性。值得注意的是，对于Qwen2.5-Omni-7B模型，OmniSIFT仅引入4.85M参数，同时保持了比OmniZip等无需训练的基线方法更低的延迟。仅使用原始令牌上下文的25%，OmniSIFT在多项任务中持续优于所有压缩基线，甚至超越了全令牌模型的性能。