Many parallel and distributed computing research results are obtained in simulation, using simulators that mimic real-world executions on some target system. Each such simulator is configured by picking values for parameters that define the behavior of the underlying simulation models it implements. The main concern for a simulator is accuracy: simulated behaviors should be as close as possible to those observed in the real-world target system. This requires that values for each of the simulator's parameters be carefully picked, or "calibrated," based on ground-truth real-world executions. Examining the current state of the art shows that simulator calibration, at least in the field of parallel and distributed computing, is often undocumented (and thus perhaps often not performed) and, when documented, is described as a labor-intensive, manual process. In this work we evaluate the benefit of automating simulation calibration using simple algorithms. Specifically, we use a real-world case study from the field of High Energy Physics and compare automated calibration to calibration performed by a domain scientist. Our main finding is that automated calibration is on par with or significantly outperforms the calibration performed by the domain scientist. Furthermore, automated calibration makes it straightforward to operate desirable trade-offs between simulation accuracy and simulation speed.

翻译：许多并行与分布式计算研究结果依赖模拟器获得，这些模拟器通过模拟目标系统上的真实执行过程来运行。每个模拟器的配置需要为其实现的底层仿真模型参数选取数值，这些参数定义了模型的行为。模拟器的核心关注点是准确性：模拟行为应尽可能接近真实目标系统中的观察结果。这要求基于真实世界的执行数据精心选取（即"校准"）模拟器各参数的数值。对当前技术现状的考察表明，至少在并行与分布式计算领域，模拟器校准往往缺乏文档记载（因此可能常未执行），即便有文档记录，也被描述为劳动密集型的人工流程。本研究通过简单算法评估自动化模拟校准的效益。具体而言，我们采用高能物理领域的真实案例研究，将自动校准与领域科学家执行的手动校准进行对比。主要发现是：自动校准的性能与领域科学家校准相当，甚至显著优于后者。此外，自动校准能便捷地实现模拟精度与模拟速度之间的理想权衡。