Many parallel and distributed computing research results are obtained in simulation, using simulators that mimic real-world executions on some target system. Each such simulator is configured by picking values for parameters that define the behavior of the underlying simulation models it implements. The main concern for a simulator is accuracy: simulated behaviors should be as close as possible to those observed in the real-world target system. This requires that values for each of the simulator's parameters be carefully picked, or "calibrated," based on ground-truth real-world executions. Examining the current state of the art shows that simulator calibration, at least in the field of parallel and distributed computing, is often undocumented (and thus perhaps often not performed) and, when documented, is described as a labor-intensive, manual process. In this work we evaluate the benefit of automating simulation calibration using simple algorithms. Specifically, we use a real-world case study from the field of High Energy Physics and compare automated calibration to calibration performed by a domain scientist. Our main finding is that automated calibration is on par with or significantly outperforms the calibration performed by the domain scientist. Furthermore, automated calibration makes it straightforward to operate desirable trade-offs between simulation accuracy and simulation speed.

翻译：许多并行与分布式计算的研究成果通过模拟获得，即使用模拟器在目标系统上复现真实世界的执行过程。每个此类模拟器均需通过选取参数值进行配置，这些参数定义了其所实现的底层模拟模型的行为。模拟器的核心关注点在于准确性：模拟行为应尽可能接近真实目标系统中观察到的行为。这要求基于真实世界的基准执行数据，为模拟器的每个参数精心选取或“校准”数值。对当前技术现状的考察表明，模拟器校准（至少在并行与分布式计算领域）往往缺乏文档记录（因而可能经常未被执行），即使有文档记录，也被描述为劳动密集型的非自动化过程。本研究通过简单算法评估自动化模拟校准的效益。具体而言，我们采用高能物理领域的真实案例，将自动校准与领域专家执行的手动校准进行对比。主要发现表明：自动校准的效果与领域专家的校准结果相当或显著更优。此外，自动校准能够直接实现模拟精度与模拟速度之间的理想权衡。