Recent advances in many domains require more and more complicated experiment design. Such complicated experiments often have many parameters, which necessitate parameter tuning. Tree-structured Parzen estimator (TPE), a Bayesian optimization method, is widely used in recent parameter tuning frameworks. Despite its popularity, the roles of each control parameter and the algorithm intuition have not been discussed so far. In this tutorial, we will identify the roles of each control parameter and their impacts on hyperparameter optimization using a diverse set of benchmarks. We compare our recommended setting drawn from the ablation study with baseline methods and demonstrate that our recommended setting improves the performance of TPE. Our TPE implementation is available at https://github.com/nabenabe0928/tpe/tree/single-opt.

翻译：近年来，许多领域的进步要求越来越复杂的实验设计。这类复杂实验通常包含大量参数，因此亟需参数调优。树结构帕森估计器（TPE）作为一种贝叶斯优化方法，已被广泛应用于最近的参数调优框架中。尽管该方法广受欢迎，但其各控制参数的作用及算法直觉至今尚未得到充分探讨。在本教程中，我们将通过一系列多样化的基准测试，识别各控制参数的作用及其对超参数优化的影响。我们将消融实验得出的推荐设置与基线方法进行比较，并证明该推荐设置能提升TPE的性能。我们的TPE实现代码已开源在 https://github.com/nabenabe0928/tpe/tree/single-opt。