AI Video Assistant emerges as a new paradigm for Real-time Communication (RTC), where one peer is a Multimodal Large Language Model (MLLM) deployed in the cloud. This makes interaction between humans and AI more intuitive, akin to chatting with a real person. However, a fundamental mismatch exists between current RTC frameworks and AI Video Assistants, stemming from the drastic shift in Quality of Experience (QoE) and more challenging networks. Measurements on our production prototype also confirm that current RTC fails, causing latency spikes and accuracy drops. To address these challenges, we propose Artic, an AI-oriented RTC framework for MLLM Video Assistants, exploring the shift from "humans watching video" to "AI understanding video." Specifically, Artic proposes: (1) Response Capability-aware Adaptive Bitrate, which utilizes MLLM accuracy saturation to proactively cap bitrate, reserving bandwidth headroom to absorb future fluctuations for latency reduction; (2) Zero-overhead Context-aware Streaming, which allocates limited bitrate to regions most important for the response, maintaining accuracy even under ultra-low bitrates; and (3) Degraded Video Understanding Benchmark, the first benchmark evaluating how RTC-induced video degradation affects MLLM accuracy. Prototype experiments using real-world uplink traces show that compared with existing methods, Artic significantly improves accuracy by 15.12% and reduces latency by 135.31 ms. We will release the benchmark and codes at https://github.com/pku-netvideo/DeViBench.

翻译：AI视频助手作为一种新型实时通信（RTC）范式应运而生，其中通信一端为部署于云端的多模态大语言模型（MLLM）。这使得人机交互更加直观，类似于与真人对话。然而，当前RTC框架与AI视频助手之间存在根本性不匹配，这源于体验质量（QoE）的剧烈转变以及更具挑战性的网络条件。对我们生产原型的实测也证实，现有RTC方案会导致延迟激增与准确率下降。为应对这些挑战，我们提出Artic——面向MLLM视频助手的AI导向RTC框架，探索从“人类观看视频”到“AI理解视频”的范式转变。具体而言，Artic提出：（1）响应能力感知的自适应码率控制，利用MLLM准确率饱和特性主动限制码率，预留带宽余量以吸收未来波动，从而降低延迟；（2）零开销上下文感知视频流传输，将有限码率分配给对响应生成最关键的区域，即使在超低码率下仍能保持准确率；（3）退化视频理解基准测试DeViBench，这是首个评估RTC所致视频退化如何影响MLLM准确率的基准。基于真实世界上行链路轨迹的原型实验表明，与现有方法相比，Artic显著提升准确率15.12%，并降低延迟135.31毫秒。我们将在https://github.com/pku-netvideo/DeViBench 开源基准测试与代码。