Long streaming video QA remains challenging due to growing visual tokens and limited reasoning length of large language models (LLMs). KV-caching stores the Key-Value (KV) of the historical tokens via LLM prefill and enables more efficient streaming QA. However, existing methods cache every one or two frames, causing redundant memory usage and losing fine-grained spatial details within frame or temporal contexts across frames. This paper proposes MuKV, a method that features a multi-grained KV cache compression module and a semi-hierarchical retrieval approach to improve both efficiency and accuracy for long streaming VideoQA. For the offline KV cache, MuKV extracts visual representations at patch-, frame-, and segment-levels. The multiple levels of granularity preserve both local cues and global temporal context, while maintaining efficiency with a dual signal token compression mechanism guided by self-attention and frequency. For online QA, MuKV designs a semi-hierarchical retrieval method to retrieve relevant KV caches for answer generation. Experiments on long-streaming VideoQA benchmarks show that MuKV significantly improves answer accuracy, without sacrificing memory and online QA efficiency. Moreover, our compression mechanism alone brings consistent benefits across answer accuracy, memory, and QA efficiency over baselines, showcasing highly effective contribution.

翻译：长流式视频问答因视觉令牌不断增长及大语言模型推理长度受限而持续面临挑战。KV缓存技术通过大语言模型预填充存储历史令牌的键值对，实现更高效的流式问答。然而现有方法对每帧或每两帧进行缓存，既导致冗余内存占用，又丢失帧内精细空间细节与帧间时间上下文信息。本文提出MuKV方法，该方法包含多粒度KV缓存压缩模块与半层次化检索策略，旨在提升长流式视频问答的效率与准确性。在离线KV缓存阶段，MuKV在补丁级、帧级和片段级三个层次提取视觉表征。多粒度层次既能保留局部线索与全局时间上下文，又通过自注意力与频率机制引导的双重信号令牌压缩技术维持高效性。在在线问答阶段，MuKV设计半层次化检索方法，检索相关KV缓存以生成答案。在长流式视频问答基准测试上的实验表明，MuKV在不牺牲内存与在线问答效率的前提下显著提升答案准确率。此外，仅压缩模块本身即可在答案准确率、内存占用与问答效率方面相较于基线方法持续获益，展现出高效贡献。