Streaming Video Large Language Models (VideoLLMs) have demonstrated impressive performance across various video understanding tasks, but they face significant challenges in real-time deployment due to the high computational cost of processing dense visual tokens from continuous video streams. In streaming video scenarios, the primary bottleneck lies in the Vision Transformer (ViT) encoding stage, where redundant processing of temporally similar frames leads to inefficiency. Additionally, inflated token sequences during LLM pre-filling further exacerbate latency and memory overhead. To address these challenges, we propose \textbf{S}treaming \textbf{T}oken \textbf{C}ompression (\textbf{STC}), a plug-and-play hierarchical framework that seamlessly integrates into existing streaming VideoLLMs, optimizing both ViT encoding and LLM pre-filling stages to accelerate processing. STC introduces two token-level accelerators: \textbf{STC-Cacher}, which reduces ViT encoding overhead by caching and reusing features from temporally similar frames, and \textbf{STC-Pruner}, which compresses the visual token sequence before it enters the LLM, preserving only the most salient tokens based on both spatial and temporal relevance. Extensive experiments on four baseline streaming VideoLLMs across five benchmarks demonstrate that STC outperforms other compression methods. Notably, STC retains up to \textbf{99\%} of accuracy on the ReKV framework while reducing ViT encoding latency and LLM pre-filling latency by \textbf{24.5\%} and \textbf{45.3\%}.

翻译：流式视频大语言模型（VideoLLMs）在各种视频理解任务中展现出卓越性能，但由于处理连续视频流中密集视觉令牌的高计算成本，其在实时部署中面临显著挑战。在流式视频场景中，主要瓶颈在于视觉Transformer（ViT）编码阶段，其中对时间相似帧的冗余处理导致效率低下。此外，LLM预填充阶段膨胀的令牌序列进一步加剧了延迟和内存开销。为应对这些挑战，我们提出**流式令牌压缩**（**STC**），一种即插即用的分层框架，可无缝集成到现有流式VideoLLMs中，通过优化ViT编码和LLM预填充阶段来加速处理。STC引入两个令牌级加速器：**STC-Cacher**通过缓存并重用时间相似帧的特征来降低ViT编码开销，以及**STC-Pruner**在视觉令牌序列输入LLM前对其进行压缩，基于空间和时间相关性仅保留最显著的令牌。在五个基准测试中对四种基线流式VideoLLMs的广泛实验表明，STC优于其他压缩方法。值得注意的是，STC在ReKV框架上保持高达**99%**的准确率，同时将ViT编码延迟和LLM预填充延迟分别降低**24.5%**和**45.3%**。