Multimodal Large Language Models (MLLMs) power platforms like ChatGPT, Gemini, and Copilot, enabling richer interactions with text, images, and videos. These heterogeneous workloads introduce additional inference stages, such as vision preprocessing and encoding, that inflate latency and memory demand. Existing LLM serving systems, optimized for text-only workloads, fail under multimodality: large requests (e.g., videos) monopolize resources, causing severe head-of-line blocking and performance degradation. Our key insight is that multimodal requests differ by orders of magnitude in resource demands, which we capture through a simple abstraction: videos behave like rocks, images like pebbles, and text like sand. We design RPS-Serve, a modality-aware scheduler that lets sand flow quickly through pebbles and rocks, ensuring interactive responsiveness while avoiding starvation. RPS-Serve classifies requests, prioritizes them dynamically, and applies aging to avoid starvation. Evaluation across state-of-the-art MLLMs shows that RPS-Serve reduces, on average, time-to-first-token (TTFT) by 54% overall, and by 78.5% for latency-critical requests, compared to current systems. RPS-Serve delivers LLM-like responsiveness for MLLMs, with modality-aware scheduling and by making the most efficient use of the available resources.

翻译：多模态大语言模型（MLLMs）支撑着ChatGPT、Gemini和Copilot等平台，实现了文本、图像和视频的丰富交互。这些异构工作负载引入了额外的推理阶段（如视觉预处理与编码），导致延迟膨胀和内存需求增加。现有针对纯文本工作负载优化的LLM服务系统在多模态场景下失效：大请求（如视频）会独占资源，引发严重的队头阻塞和性能退化。我们的核心洞察在于，多模态请求的资源需求相差数个数量级，并通过一个简洁的抽象加以捕捉：视频如同石头、图像如同卵石、文本如同沙子。我们设计了RPS-Serve——一种模态感知调度器，让沙子快速流过卵石和石头，在避免饥饿的同时保证交互式响应能力。RPS-Serve对请求进行分类、动态调整优先级，并采用老化机制防止饥饿。对最新MLLM的评估表明，与现有系统相比，RPS-Serve平均将首令牌时间（TTFT）降低54%，对延迟敏感请求则降低78.5%。RPS-Serve通过模态感知调度并高效利用可用资源，为MLLM实现了类似LLM的响应能力。