Recent advances in many domains require more and more complicated experiment design. Such complicated experiments often have many parameters, which necessitate parameter tuning. Tree-structured Parzen estimator (TPE), a Bayesian optimization method, is widely used in recent parameter tuning frameworks. Despite its popularity, the roles of each control parameter and the algorithm intuition have not been discussed so far. In this tutorial, we will identify the roles of each control parameter and their impacts on hyperparameter optimization using a diverse set of benchmarks. We compare our recommended setting drawn from the ablation study with baseline methods and demonstrate that our recommended setting improves the performance of TPE. Our TPE implementation is available at https://github.com/nabenabe0928/tpe/tree/single-opt.

翻译：近期众多领域的进步需要越来越复杂的实验设计。此类复杂实验通常具有大量参数，因而需要参数调优。树结构Parzen估计器（TPE）作为一种贝叶斯优化方法，被广泛应用于当前的参数调优框架中。尽管其应用广泛，但各控制参数的作用及算法直觉尚未得到充分讨论。在本教程中，我们将利用多样化的基准测试，明确各控制参数的角色及其对超参数优化的影响。我们将消融研究得出的推荐设置与基线方法进行对比，并证明我们的推荐设置能提升TPE的性能。我们的TPE实现代码已开源至https://github.com/nabenabe0928/tpe/tree/single-opt。