Many state-of-the-art hyperparameter optimization (HPO) algorithms rely on model-based optimizers that learn surrogate models of the target function to guide the search. Gaussian processes are the de facto surrogate model due to their ability to capture uncertainty but they make strong assumptions about the observation noise, which might not be warranted in practice. In this work, we propose to leverage conformalized quantile regression which makes minimal assumptions about the observation noise and, as a result, models the target function in a more realistic and robust fashion which translates to quicker HPO convergence on empirical benchmarks. To apply our method in a multi-fidelity setting, we propose a simple, yet effective, technique that aggregates observed results across different resource levels and outperforms conventional methods across many empirical tasks.

翻译：许多最先进的超参数优化（HPO）算法依赖于基于模型的优化器，这些优化器通过学习目标函数的代理模型来指导搜索过程。高斯过程因其捕捉不确定性的能力而成为事实上的代理模型，但它们对观测噪声做出了强烈假设，而这些假设在实践中可能难以成立。在这项工作中，我们提出利用共形化分位数回归方法，该方法对观测噪声的假设极少，从而能够以更真实且稳健的方式对目标函数建模，进而提升HPO在经验基准测试中的收敛速度。为了在多保真度场景下应用我们的方法，我们提出了一种简单而有效的技术，该技术聚合不同资源层级上的观测结果，并在多项经验任务中优于传统方法。