Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently. While prior work has largely focused on latency minimization, resource efficiency through throughput maximization remains underexplored. This paper presents a data-driven pipeline that, for a given workload, computes an adapter placement that serves the workload with the minimum number of GPUs while avoiding request starvation and GPU memory errors. To that end, the approach identifies the maximum feasible throughput attainable on each GPU by leveraging accurate performance predictions learned from real serving behavior. The proposed pipeline integrates three components: (i) a Digital Twin (DT) tailored to LLM-adapter serving, (ii) a distilled machine learning (ML) model trained on DT-generated data, and (iii) a greedy placement algorithm that exploits ML-based performance estimates to maximize GPU efficiency. The DT emulates real system dynamics with high fidelity, achieving below 5% throughput estimation error while executing up to 90 times faster than full LLM benchmarking across both predictable and unpredictable workloads. The learned ML models further accelerate performance estimation with marginal accuracy degradation, enabling scalable optimization. Experimental results demonstrate that the pipeline substantially improves GPU efficiency by reducing the number of GPUs required to sustain target workloads. Beyond GPU efficiency, the pipeline can be adapted to alternative objectives, such as latency minimization, highlighting its versatility for future large-scale LLM serving infrastructures.

翻译：大型语言模型（LLM）适配器能够以低成本实现模型专业化，但在需要同时托管数百个适配器的分布式服务系统中，引入了复杂的缓存与调度挑战。现有研究主要集中于延迟最小化，而通过吞吐量最大化实现的资源效率仍未得到充分探索。本文提出一种数据驱动的流程，针对给定工作负载，计算一种适配器放置方案，使其能够以最少的GPU数量服务该工作负载，同时避免请求饥饿和GPU内存错误。为此，该方法通过利用从真实服务行为中学习到的精确性能预测，识别每个GPU上可达到的最大可行吞吐量。所提出的流程整合了三个组件：（i）专为LLM适配器服务定制的数字孪生（DT），（ii）基于DT生成数据训练的蒸馏机器学习（ML）模型，以及（iii）一种利用基于ML的性能估计来最大化GPU效率的贪心放置算法。该数字孪生以高保真度模拟真实系统动态，在可预测和不可预测的工作负载下，其吞吐量估计误差低于5%，且执行速度比完整的LLM基准测试快达90倍。学习得到的ML模型进一步加速了性能估计，同时精度损失极小，实现了可扩展的优化。实验结果表明，该流程通过减少维持目标工作负载所需的GPU数量，显著提升了GPU效率。除GPU效率外，该流程还可适应其他目标（如延迟最小化），突显了其对于未来大规模LLM服务基础设施的通用性。