Data center schedulers operate at unprecedented scales today to accommodate the growing demand for computing and storage power. The challenge that schedulers face is meeting the requirements of scheduling speeds despite the scale. To do so, most scheduler architectures use parallelism. However, these architectures consist of multiple parallel scheduling entities that can only utilize partial knowledge of the data center's state, as maintaining consistent global knowledge or state would involve considerable communication overhead. The disadvantage of scheduling without global knowledge is sub-optimal placements-tasks may be made to wait in queues even though there are resources available in zones outside the scope of the scheduling entity's state. This leads to unnecessary queuing overheads and lower resource utilization of the data center. In this paper, extend our previous work on Megha, a federated decentralized data center scheduling architecture that uses eventual consistency. The architecture utilizes both parallelism and an eventually-consistent global state in each of its scheduling entities to make fast decisions in a scalable manner. In our work, we compare Megha with 3 scheduling architectures: Sparrow, Eagle, and Pigeon, using simulation. We also evaluate Megha's prototype on a 123-node cluster and compare its performance with Pigeon's prototype using cluster traces. The results of our experiments show that Megha consistently reduces delays in job completion time when compared to other architectures.

翻译：数据中心调度器如今以前所未有的规模运行，以满足日益增长的计算和存储能力需求。调度器面临的挑战是在规模化的同时满足调度速度的要求。为此，大多数调度器架构采用并行化方式。然而，这些架构由多个并行调度实体组成，每个实体只能利用数据中心的局部状态信息，因为维护全局一致性状态需要巨大的通信开销。缺乏全局知识的调度会导致次优的任务放置——即使调度实体状态范围之外的区域存在可用资源，任务也可能被迫在队列中等待。这带来了不必要的排队开销，并降低了数据中心的资源利用率。本文扩展了我们先前关于Megha的工作，这是一种采用最终一致性的联邦式去中心化数据中心调度架构。该架构在每个调度实体中同时利用并行性和最终一致性全局状态，以可扩展的方式快速做出决策。我们通过仿真将Megha与三种调度架构（Sparrow、Eagle和Pigeon）进行了比较。同时，我们在123节点集群上评估了Megha的原型，并使用集群跟踪数据将其性能与Pigeon原型进行了对比。实验结果表明，与其他架构相比，Megha始终能减少作业完成时间的延迟。