Video streaming analytics is a crucial workload for vision-language model serving, but the high cost of multimodal inference limits scalability. Prior systems reduce inference cost by exploiting temporal and spatial redundancy in video streams, but they target either the vision transformer (ViT) or the LLM with a limited view, leaving end-to-end opportunities untapped. Moreover, existing methods incur significant overhead to identify redundancy, either through offline profiling and training or costly online computation, making them ill-suited for dynamic real-time streams. We present CodecFlow, a codec-guided streaming video analytics system built on a key observation that video codecs already extract the temporal and spatial structure of each stream as a byproduct of compression. CodecFlow treats this codec metadata as a low-cost runtime signal to unify optimization across video decoding, visual processing, and LLM prefilling, with transmission reduction as an inherent benefit of operating directly on compressed bitstreams. This drives codec-guided patch pruning before ViT encoding and selective key-value cache refresh during LLM prefilling, both of which are fully online and do not require offline training. Experiments show that CodecFlow achieves up to 3x throughput improvement and up to 87% GPU compute reduction over state-of-the-art baselines, while maintaining competitive accuracy with only 0-8% F1 drop.

翻译：流式视频分析是视觉-语言模型服务中的关键负载，但多模态推理的高昂成本限制了其可扩展性。现有系统通过利用视频流中的时空冗余来降低推理成本，但它们仅针对视觉Transformer或大语言模型进行局部优化，未能充分挖掘端到端优化潜力。此外，现有方法需要通过离线性能分析训练或高开销在线计算来识别冗余，导致其难以适应动态实时流场景。我们提出CodecFlow——一种基于编解码器引导的流式视频分析系统，其核心发现是：视频编解码器在压缩过程中已天然提取了每个流的时空结构。CodecFlow将这种编解码元数据作为低开销运行时信号，统一优化视频解码、视觉处理与大语言模型预填充（prefilling），而直接操作压缩比特流的特性则天然支持传输开销缩减。该系统在视觉Transformer编码前实现编解码器引导的补丁剪枝，并在大语言模型预填充期间执行选择性键值缓存刷新，两者均为全在线操作且无需离线训练。实验表明：相比最先进基线系统，CodecFlow可实现最高3倍的吞吐量提升、最高87%的GPU计算量降低，同时保持竞争性精度（F1分数仅下降0-8%）。