Uncertainty Quantification (UQ) workloads are becoming increasingly common in science and engineering. They involve the submission of thousands or even millions of similar tasks with potentially unpredictable runtimes, where the total number is usually not known a priori. A static one-size-fits-all batch script would likely lead to suboptimal scheduling, and native schedulers installed on High Performance Computing (HPC) systems such as SLURM often struggle to efficiently handle such workloads. In this paper, we introduce a new load balancing approach suitable for UQ workflows. To demonstrate its efficiency in a real-world setting, we focus on the GS2 gyrokinetic plasma turbulence simulator. Individual simulations can be computationally demanding, with runtimes varying significantly-from minutes to hours-depending on the high-dimensional input parameters. Our approach uses UQ and Modelling Bridge, which offers a language-agnostic interface to a simulation model, combined with HyperQueue which works alongside the native scheduler. In particular, deploying this framework on HPC systems does not require system-level changes. We benchmark our proposed framework against a standalone SLURM approach using GS2 and a Gaussian Process surrogate thereof. Our results demonstrate a reduction in scheduling overhead by up to three orders of magnitude and a maximum reduction of 38% in CPU time for long-running simulations compared to the naive SLURM approach, while making no assumptions about the job submission patterns inherent to UQ workflows.

翻译：不确定性量化（UQ）工作负载在科学与工程领域日益普遍。这类工作负载通常涉及提交数千甚至数百万个具有相似性但运行时间可能无法预测的任务，且任务总数通常无法预先获知。采用静态的通用批量脚本很可能导致次优调度，而高性能计算（HPC）系统（如SLURM）上安装的原生调度器往往难以高效处理此类工作负载。本文提出一种适用于UQ工作流的新型负载均衡方法。为在实际场景中验证其效率，我们以GS2回旋动能等离子体湍流模拟器为研究对象。单个模拟任务可能计算量巨大，其运行时间根据高维输入参数的不同可在数分钟至数小时范围内显著波动。我们的方法结合了UQ与建模桥接器（提供与模拟模型无关的语言接口）和HyperQueue（与原生调度器协同工作）。特别值得注意的是，该框架在HPC系统上的部署无需系统级修改。我们使用GS2及其高斯过程代理模型，将所提框架与独立SLURM方法进行基准测试。结果表明：相较于原生SLURM方法，该框架将调度开销降低了高达三个数量级，并为长时间运行的模拟任务最多减少了38%的CPU时间，同时无需对UQ工作流固有的作业提交模式做任何假设。