With continuous advances in deep learning, distributed training is becoming common in GPU clusters. Specifically, for emerging workloads with diverse amounts, ratios, and patterns of communication, we observe that network contention can significantly degrade training throughput. However, widely used scheduling policies often face limitations as they are agnostic to network contention between jobs. In this paper, we present a new approach to mitigate network contention in GPU clusters using reinforcement learning. We formulate GPU cluster scheduling as a reinforcement learning problem and opt to learn a network contention-aware scheduling policy that efficiently captures contention sensitivities and dynamically adapts scheduling decisions through continuous evaluation and improvement. We show that compared to widely used scheduling policies, our approach reduces average job completion time by up to 18.2\% and effectively cuts the tail job completion time by up to 20.7\% while allowing a preferable trade-off between average job completion time and resource utilization.

翻译：随着深度学习的不断进步，分布式训练在GPU集群中已变得普遍。具体而言，对于通信量、通信比例及通信模式各异的新兴工作负载，我们观察到网络争用会显著降低训练吞吐量。然而，广泛采用的调度策略由于对作业间的网络争用不敏感而经常面临局限性。本文提出了一种利用强化学习缓解GPU集群中网络争用的新方法。我们将GPU集群调度建模为强化学习问题，并致力于学习一种网络争用感知的调度策略，该策略能有效捕捉争用敏感性，并通过持续评估与改进动态调整调度决策。研究表明，与广泛使用的调度策略相比，我们的方法可将平均作业完成时间降低最高18.2%，尾部作业完成时间降低最高20.7%，同时在平均作业完成时间与资源利用率之间实现更优的权衡。