Scheduling a task graph representing an application over a heterogeneous network of computers is a fundamental problem in distributed computing. It is known to be not only NP-hard but also not polynomial-time approximable within a constant factor. As a result, many heuristic algorithms have been proposed over the past few decades. Yet it remains largely unclear how these algorithms compare to each other in terms of the quality of schedules they produce. We identify gaps in the traditional benchmarking approach to comparing task scheduling algorithms and propose a simulated annealing-based adversarial analysis approach called PISA to help address them. We also introduce SAGA, a new open-source library for comparing task scheduling algorithms. We use SAGA to benchmark 15 algorithms on 16 datasets and PISA to compare the algorithms in a pairwise manner. Algorithms that appear to perform similarly on benchmarking datasets are shown to perform very differently on adversarially chosen problem instances. Interestingly, the results indicate that this is true even when the adversarial search is constrained to selecting among well-structured, application-specific problem instances. This work represents an important step towards a more general understanding of the performance boundaries between task scheduling algorithms on different families of problem instances.

翻译：在异构计算机网络上调度表示应用程序的任务图是分布式计算中的一个基本问题。已知该问题不仅是NP难问题，而且无法在常数因子内进行多项式时间近似求解。因此，过去几十年间提出了许多启发式算法。然而，这些算法在生成调度方案的质量方面如何相互比较，在很大程度上仍不清楚。我们指出了比较任务调度算法的传统基准测试方法中存在的不足，并提出了一种基于模拟退火的对抗性分析方法PISA以帮助解决这些问题。同时，我们引入了SAGA——一个用于比较任务调度算法的新开源库。我们使用SAGA在16个数据集上对15种算法进行基准测试，并利用PISA对这些算法进行两两比较。在基准测试数据集上表现相似的算法，在对抗性选择的问题实例上却表现出显著差异。有趣的是，即使对抗性搜索被限制在结构良好、特定于应用的问题实例中进行选择，这一结论仍然成立。这项工作代表了朝着更全面理解任务调度算法在不同问题实例族上的性能边界迈出的重要一步。