Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. In most current research into heuristic optimization algorithms, only a very limited number of scenarios, algorithm configurations and hyper-parameter settings are explored, leading to incomplete and often biased insights and results. This paper presents a novel approach we call explainable benchmarking. Introducing the IOH-Xplainer software framework, for analyzing and understanding the performance of various optimization algorithms and the impact of their different components and hyper-parameters. We showcase the framework in the context of two modular optimization frameworks. Through this framework, we examine the impact of different algorithmic components and configurations, offering insights into their performance across diverse scenarios. We provide a systematic method for evaluating and interpreting the behaviour and efficiency of iterative optimization heuristics in a more transparent and comprehensible manner, allowing for better benchmarking and algorithm design.

翻译：基准测试启发式算法对于理解算法在何种条件下、针对哪类问题表现良好至关重要。在目前大多数关于启发式优化算法的研究中，仅探索了极其有限的场景、算法配置和超参数设置，导致产生了不完整且往往带有偏见的洞察和结果。本文提出了一种我们称之为可解释基准测试的新方法。我们引入了IOH-Xplainer软件框架，用于分析和理解各种优化算法的性能，以及其不同组件和超参数的影响。我们以两个模块化优化框架为背景展示了该框架。通过这一框架，我们考察了不同算法组件和配置的影响，提供了它们在多种场景下性能表现的洞见。我们提供了一种系统方法，以更透明和更易理解的方式评估和解释迭代优化启发式算法的行为与效率，从而实现了更好的基准测试和算法设计。