The performance of modern machine learning algorithms depends upon the selection of a set of hyperparameters. Common examples of hyperparameters are learning rate and the number of layers in a dense neural network. Auto-ML is a branch of optimization that has produced important contributions in this area. Within Auto-ML, hyperband-based approaches, which eliminate poorly-performing configurations after evaluating them at low budgets, are among the most effective. However, the performance of these algorithms strongly depends on how effectively they allocate the computational budget to various hyperparameter configurations. We present the new Parameter Optimization with Conscious Allocation (POCA), a hyperband-based algorithm that adaptively allocates the inputted budget to the hyperparameter configurations it generates following a Bayesian sampling scheme. We compare POCA to its nearest competitor at optimizing the hyperparameters of an artificial toy function and a deep neural network and find that POCA finds strong configurations faster in both settings.

翻译：现代机器学习算法的性能依赖于一组超参数的选择。常见的超参数示例包括学习率和密集神经网络中的层数。自动机器学习是优化领域的一个分支，在该领域已取得重要成果。在自动机器学习中，基于超带的算法最为有效，这些算法在低计算预算下评估超参数配置后，会淘汰表现不佳的配置。然而，这些算法的性能在很大程度上取决于它们如何有效地将计算预算分配给不同的超参数配置。我们提出了一种新的有意识分配参数优化算法（POCA），这是一种基于超带的算法，能够自适应地将输入预算按照贝叶斯采样方案分配给其生成的超参数配置。我们将POCA与其最接近的竞争算法在优化人工玩具函数和深度神经网络的超参数方面进行了比较，发现POCA在两种场景下都能更快地找到强配置。