Despite all the benefits of automated hyperparameter optimization (HPO), most modern HPO algorithms are black-boxes themselves. This makes it difficult to understand the decision process which leads to the selected configuration, reduces trust in HPO, and thus hinders its broad adoption. Here, we study the combination of HPO with interpretable machine learning (IML) methods such as partial dependence plots. These techniques are more and more used to explain the marginal effect of hyperparameters on the black-box cost function or to quantify the importance of hyperparameters. However, if such methods are naively applied to the experimental data of the HPO process in a post-hoc manner, the underlying sampling bias of the optimizer can distort interpretations. We propose a modified HPO method which efficiently balances the search for the global optimum w.r.t. predictive performance \emph{and} the reliable estimation of IML explanations of an underlying black-box function by coupling Bayesian optimization and Bayesian Algorithm Execution. On benchmark cases of both synthetic objectives and HPO of a neural network, we demonstrate that our method returns more reliable explanations of the underlying black-box without a loss of optimization performance.

翻译：尽管自动化超参数优化（HPO）具有诸多益处，但现代HPO算法本身多为黑箱模型。这使得难以理解导致选定配置的决策过程，降低了对HPO的信任度，从而阻碍了其广泛应用。本文研究将HPO与可解释机器学习（IML）方法（如部分依赖图）相结合。这些技术越来越多地被用于解释超参数对黑箱代价函数的边际效应，或量化超参数的重要性。然而，若事后将此类方法简单应用于HPO过程的实验数据，优化器固有的采样偏差可能会扭曲解释结果。我们提出一种改进的HPO方法，通过耦合贝叶斯优化与贝叶斯算法执行，在寻求全局最优预测性能与可靠估计底层黑箱函数的IML解释之间实现高效权衡。在合成目标函数和神经网络HPO的基准案例中，我们证明该方法能在不牺牲优化性能的前提下，返回更可靠的底层黑箱解释。