Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. In most current research into heuristic optimization algorithms, only a very limited number of scenarios, algorithm configurations and hyper-parameter settings are explored, leading to incomplete and often biased insights and results. This paper presents a novel approach we call explainable benchmarking. Introducing the IOH-Xplainer software framework, for analyzing and understanding the performance of various optimization algorithms and the impact of their different components and hyper-parameters. We showcase the framework in the context of two modular optimization frameworks. Through this framework, we examine the impact of different algorithmic components and configurations, offering insights into their performance across diverse scenarios. We provide a systematic method for evaluating and interpreting the behaviour and efficiency of iterative optimization heuristics in a more transparent and comprehensible manner, allowing for better benchmarking and algorithm design.

翻译：对启发式算法进行基准测试，对于理解特定算法在何种条件及何种问题上表现良好至关重要。在目前大多数启发式优化算法的研究中，仅探索了非常有限的场景、算法配置和超参数设置，导致结论不完整且常常存在偏差。本文提出了一种称为可解释基准测试的新方法。我们介绍了IOH-Xplainer软件框架，用于分析和理解各种优化算法的性能，及其不同组件和超参数的影响。我们以两个模块化优化框架为例，展示了该框架的应用。通过此框架，我们考察了不同算法组件和配置的影响，揭示了它们在不同场景下的性能表现。我们提供了一种系统化的方法，能够以更透明、更易理解的方式评估和解释迭代优化启发式算法的行为与效率，从而实现更优的基准测试和算法设计。