Hyper-parameter optimization is one of the most tedious yet crucial steps in training machine learning models. There are numerous methods for this vital model-building stage, ranging from domain-specific manual tuning guidelines suggested by the oracles to the utilization of general-purpose black-box optimization techniques. This paper proposes an agent-based collaborative technique for finding near-optimal values for any arbitrary set of hyper-parameters (or decision variables) in a machine learning model (or general function optimization problem). The developed method forms a hierarchical agent-based architecture for the distribution of the searching operations at different dimensions and employs a cooperative searching procedure based on an adaptive width-based random sampling technique to locate the optima. The behavior of the presented model, specifically against the changes in its design parameters, is investigated in both machine learning and global function optimization applications, and its performance is compared with that of two randomized tuning strategies that are commonly used in practice. According to the empirical results, the proposed model outperformed the compared methods in the experimented classification, regression, and multi-dimensional function optimization tasks, notably in a higher number of dimensions and in the presence of limited on-device computational resources.

翻译：超参数优化是训练机器学习模型中最繁琐但至关重要的步骤之一。针对这一关键模型构建阶段，存在多种方法，从领域专家提出的特定领域手动调优指南，到通用黑箱优化技术的应用。本文提出了一种基于智能体的协作技术，用于在机器学习模型（或一般函数优化问题）中寻找任意超参数集（或决策变量）的近似最优值。该方法构建了分层智能体架构，以在不同维度上分布搜索操作，并采用基于自适应宽度随机采样技术的协作搜索流程来定位最优解。本文在机器学习与全局函数优化应用中研究了所提模型的行为（特别是其设计参数变化的影响），并将其性能与两种实际常用的随机调优策略进行了比较。实验结果表明，所提模型在分类、回归及多维函数优化任务中优于对比方法，尤其在更高维度及设备端计算资源有限的场景下表现突出。