The unknown parameters of simulation models often need to be calibrated using observed data. When simulation models are expensive, calibration is usually carried out with an emulator. The effectiveness of the calibration process can be significantly improved by using a sequential selection of parameters to build an emulator. The expansion of parallel computing environments--from multicore personal computers to many-node servers to large-scale cloud computing environments--can lead to further calibration efficiency gains by allowing for the evaluation of the simulation model at a batch of parameters in parallel in a sequential design. However, understanding the performance implications of different sequential approaches in parallel computing environments introduces new complexities since the rate of the speed-up is affected by many factors, such as the run time of a simulation model and the variability in the run time. This work proposes a new performance model to understand and benchmark the performance of different sequential procedures for the calibration of simulation models in parallel environments. We provide metrics and a suite of techniques for visualizing the numerical experiment results and demonstrate these with a novel sequential procedure. The proposed performance model, as well as the new sequential procedure and other state-of-art techniques, are implemented in the open-source Python software package Parallel Uncertainty Quantification (PUQ), which allows users to run a simulation model in parallel.

翻译：仿真模型的未知参数通常需要利用观测数据进行校准。当仿真模型计算成本高昂时，校准过程通常借助代理模型完成。通过序列化选择参数构建代理模型，可显著提升校准过程的有效性。并行计算环境的发展——从多核个人计算机到多节点服务器，再到大规模云计算环境——使得在序列化设计中能够并行评估一批参数，从而进一步提升校准效率。然而，理解并行计算环境中不同序列化方法的性能影响带来了新的复杂性，因为加速比受多种因素影响，例如仿真模型的运行时间及其变异性。本研究提出一种新的性能模型，用以理解和评估并行环境下仿真模型校准的不同序列化流程的性能表现。我们提供了一套度量指标与可视化数值实验结果的综合方法，并通过一种新型序列化流程进行演示。所提出的性能模型、新型序列化流程及其他前沿技术均已集成于开源Python软件包Parallel Uncertainty Quantification（PUQ）中，该工具支持用户在并行环境中运行仿真模型。