Diffusion models have emerged as the prevailing approach for text-to-image (T2I) and text-to-video (T2V) generation, yet production platforms must increasingly serve both modalities on shared GPU clusters while meeting stringent latency SLOs. Co-serving such heterogeneous workloads is challenging: T2I and T2V requests exhibit vastly different compute demands, parallelism characteristics, and latency requirements, leading to significant SLO violations in existing serving systems. We present GENSERVE, a co-serving system that leverages the inherent predictability of the diffusion process to optimize serving efficiency. A central insight is that diffusion inference proceeds in discrete, predictable steps and is naturally preemptible at step boundaries, opening a new design space for heterogeneity-aware resource management. GENSERVE introduces step-level resource adaptation through three coordinated mechanisms: intelligent video preemption, elastic sequence parallelism with dynamic batching, and an SLO-aware scheduler that jointly optimizes resource allocation across all concurrent requests. Experimental results show that GENSERVE improves the SLO attainment rate by up to 44% over the strongest baseline across diverse configurations.

翻译：扩散模型已成为文本到图像（T2I）和文本到视频（T2V）生成的主流方法，然而生产平台必须在共享GPU集群上同时服务这两种模态，并满足严格的延迟服务等级协议（SLO）。协同服务此类异构工作负载颇具挑战：T2I和T2V请求在计算需求、并行特性及延迟要求上存在显著差异，导致现有服务系统出现严重的SLO违反问题。我们提出GENSERVE——一种利用扩散过程内在可预测性来优化服务效率的协同服务系统。其核心洞察在于：扩散推理以离散、可预测的步骤进行，并天然可在步骤边界处被抢占，这为异构感知资源管理开辟了新的设计空间。GENSERVE通过三种协同机制引入步骤级资源自适应：智能视频抢占、弹性序列并行与动态批处理，以及一个面向SLO感知的调度器，可联合优化所有并发请求的资源分配。实验结果表明，在多种配置下，GENSERVE相比最强基线将SLO达标率提升高达44%。