Text-to-image (T2I) models have gained significant popularity. Most of these are diffusion models with unique computational characteristics, distinct from both traditional small-scale ML models and large language models. They are highly compute-bound and use an iterative denoising process to generate images, leading to very high inference time. This creates significant challenges in designing a high-throughput system. We discovered that a large fraction of prompts can be served using faster, approximated models. However, the approximation setting must be carefully calibrated for each prompt to avoid quality degradation. Designing a high-throughput system that assigns each prompt to the appropriate model and compatible approximation setting remains a challenging problem. We present Argus, a high-throughput T2I inference system that selects the right level of approximation for each prompt to maintain quality while meeting throughput targets on a fixed-size cluster. Argus intelligently switches between different approximation strategies to satisfy both throughput and quality requirements. Overall, Argus achieves 10x fewer latency service-level objective (SLO) violations, 10% higher average quality, and 40% higher throughput compared to baselines on two real-world workload traces.

翻译：文本到图像（T2I）模型已获得广泛关注。其中大多数为扩散模型，具有独特的计算特性，既不同于传统的小规模机器学习模型，也不同于大型语言模型。这些模型计算密集度高，采用迭代去噪过程生成图像，导致推理时间极长，这为设计高吞吐量系统带来了显著挑战。我们发现，大量提示词可通过更快速的近似模型进行处理。然而，必须针对每个提示词仔细校准近似设置，以避免生成质量下降。设计一个能将每个提示词分配至合适模型及兼容近似设置的高吞吐量系统，仍是一个具有挑战性的问题。本文提出Argus，一种高吞吐量的T2I推理系统，它能为每个提示词选择合适的近似级别，在满足固定规模集群吞吐量目标的同时保持生成质量。Argus智能地在不同近似策略间切换，以满足吞吐量与质量的双重要求。总体而言，在两种真实工作负载轨迹上，与基线方法相比，Argus实现了延迟服务级别目标（SLO）违反次数减少10倍、平均质量提升10%、吞吐量提高40%的性能表现。