We present an analysis of large-scale load balancing systems, where the processing time distribution of tasks depends on both the task and server types. Our study focuses on the asymptotic regime, where the number of servers and task types tend to infinity in proportion. In heterogeneous environments, commonly used load balancing policies such as Join Fastest Idle Queue and Join Fastest Shortest Queue exhibit poor performance and even shrink the stability region. Interestingly, prior to this work, finding a scalable policy with a provable performance guarantee in this setup remained an open question. To address this gap, we propose and analyze two asymptotically delay-optimal dynamic load balancing policies. The first policy efficiently reserves the processing capacity of each server for ``good" tasks and routes tasks using the vanilla Join Idle Queue policy. The second policy, called the speed-priority policy, significantly increases the likelihood of assigning tasks to the respective ``good" servers capable of processing them at high speeds. By leveraging a framework inspired by the graphon literature and employing the mean-field method and stochastic coupling arguments, we demonstrate that both policies achieve asymptotic zero queuing. Specifically, as the system scales, the probability of a typical task being assigned to an idle server approaches 1.

翻译：我们分析了大尺度负载均衡系统，其中任务处理时间分布同时依赖于任务类型和服务器类型。研究聚焦于渐近框架，即服务器数量与任务类型数按比例趋于无穷大。在异构环境下，常用负载均衡策略（如Join Fastest Idle Queue和Join Fastest Shortest Queue）性能表现欠佳，甚至导致稳定区域收缩。值得注意的是，在本文之前，针对该场景寻找具有可证明性能保证的可扩展策略一直是个开放性问题。为填补这一空白，我们提出并分析了两种渐近时延最优的动态负载均衡策略。第一种策略通过高效保留每台服务器对“优质”任务的处理容量，并采用原始Join Idle Queue策略路由任务；第二种策略称为“速度优先级策略”，显著提升将任务分配给能高速处理该任务的“优质”服务器的概率。利用图论文献启发的框架，结合均场方法与随机耦合论证，我们证明两种策略均能实现渐近零排队。具体而言，随着系统规模扩展，典型任务被分配给空闲服务器的概率趋近于1。