Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated significant improvement in offline video understanding. However, extending these capabilities to streaming video inputs, remains challenging, as existing models struggle to simultaneously maintain stable understanding performance, real-time responses, and low GPU memory overhead. To address this challenge, we propose HERMES, a novel training-free architecture for real-time and accurate understanding of video streams. Based on a mechanistic attention investigation, we conceptualize KV cache as a hierarchical memory framework that encapsulates video information across multiple granularities. During inference, HERMES reuses a compact KV cache, enabling efficient streaming understanding under resource constraints. Notably, HERMES requires no auxiliary computations upon the arrival of user queries, thereby guaranteeing real-time responses for continuous video stream interactions, which achieves 10$\times$ faster TTFT compared to prior SOTA. Even when reducing video tokens by up to 68% compared with uniform sampling, HERMES achieves superior or comparable accuracy across all benchmarks, with up to 11.4% gains on streaming datasets.

翻译：近年来，多模态大语言模型（MLLMs）在离线视频理解方面取得了显著进展。然而，将其能力扩展到流式视频输入仍然具有挑战性，因为现有模型难以同时维持稳定的理解性能、实时响应以及较低的GPU内存开销。为应对这一挑战，我们提出了HERMES，一种无需训练的新型架构，用于实时且准确地理解视频流。基于对注意力机制的机理性研究，我们将KV缓存概念化为一个层级化内存框架，该框架以多粒度封装视频信息。在推理过程中，HERMES复用一个紧凑的KV缓存，从而在资源受限条件下实现高效的流式理解。值得注意的是，HERMES在用户查询到达时无需任何辅助计算，从而保证了连续视频流交互的实时响应，其首词生成时间（TTFT）较先前SOTA方法快10倍。即使在视频令牌数量相比均匀采样减少高达68%的情况下，HERMES在所有基准测试中仍实现了更优或相当的性能，在流式数据集上最高可获得11.4%的性能提升。