Programmatic hyperparameter optimization (HPO) methods, such as Bayesian optimization and evolutionary algorithms, are highly sample-efficient in identifying optimal hyperparameter configurations for machine learning (ML) models. However, practitioners frequently use less efficient methods, such as grid search, which can lead to under-optimized models. We suspect this behavior is driven by a range of practitioner-specific motives. Practitioner motives, however, still need to be clarified to enhance user-centered development of HPO tools. To uncover practitioner motives to use different HPO methods, we conducted 20 semi-structured interviews and an online survey with 49 ML experts. By presenting main goals (e.g., increase ML model understanding) and contextual factors affecting practitioners' selection of HPO methods (e.g., available computer resources), this study offers a conceptual foundation to better understand why practitioners use different HPO methods, supporting development of more user-centered and context-adaptive HPO tools in automated ML.
翻译:程序化超参数优化(HPO)方法,如贝叶斯优化和进化算法,在识别机器学习(ML)模型最优超参数配置方面具有极高的样本效率。然而,从业者经常使用效率较低的方法,如网格搜索,这可能导致模型优化不足。我们推测这种行为是由一系列从业者特定的动机驱动的。然而,为了增强以用户为中心的HPO工具开发,从业者的动机仍需阐明。为了揭示从业者选择不同HPO方法的动机,我们进行了20次半结构化访谈,并对49位ML专家进行了在线调查。通过呈现影响从业者选择HPO方法的主要目标(例如,增进ML模型理解)和情境因素(例如,可用的计算资源),本研究为更好地理解从业者为何使用不同的HPO方法提供了概念基础,从而支持在自动化ML中开发更具用户中心性和情境适应性的HPO工具。