Machine learning algorithms have been used widely in various applications and areas. To fit a machine learning model into different problems, its hyper-parameters must be tuned. Selecting the best hyper-parameter configuration for machine learning models has a direct impact on the model's performance. It often requires deep knowledge of machine learning algorithms and appropriate hyper-parameter optimization techniques. Although several automatic optimization techniques exist, they have different strengths and drawbacks when applied to different types of problems. In this paper, optimizing the hyper-parameters of common machine learning models is studied. We introduce several state-of-the-art optimization techniques and discuss how to apply them to machine learning algorithms. Many available libraries and frameworks developed for hyper-parameter optimization problems are provided, and some open challenges of hyper-parameter optimization research are also discussed in this paper. Moreover, experiments are conducted on benchmark datasets to compare the performance of different optimization methods and provide practical examples of hyper-parameter optimization. This survey paper will help industrial users, data analysts, and researchers to better develop machine learning models by identifying the proper hyper-parameter configurations effectively.

翻译：机器学习算法被广泛应用于各种应用和领域。为了将机器学习模型适应不同的问题,必须调整其超参数。为机器学习模型选择最佳超参数配置对模型的性能有直接影响。它往往需要深入了解机器学习算法和适当的超参数优化技术。虽然存在几种自动优化技术,但应用到不同类型的问题时它们有不同的优点和缺点。本文研究的是优化通用机器学习模型的超参数。我们引入了若干最先进的优化技术,并讨论如何将其应用于机器学习算法。我们提供了许多为超参数优化问题开发的图书馆和框架,本文也讨论了超参数优化研究的一些公开挑战。此外,还在基准数据集上进行了实验,以比较不同优化方法的性能,并提供超参数优化的实用实例。这份调查文件将有助于工业用户、数据分析员和研究人员通过有效地确定适当的超参数配置更好地开发机器学习模型。