We introduce ordered transfer hyperparameter optimisation (OTHPO), a version of transfer learning for hyperparameter optimisation (HPO) where the tasks follow a sequential order. Unlike for state-of-the-art transfer HPO, the assumption is that each task is most correlated to those immediately before it. This matches many deployed settings, where hyperparameters are retuned as more data is collected; for instance tuning a sequence of movie recommendation systems as more movies and ratings are added. We propose a formal definition, outline the differences to related problems and propose a basic OTHPO method that outperforms state-of-the-art transfer HPO. We empirically show the importance of taking order into account using ten benchmarks. The benchmarks are in the setting of gradually accumulating data, and span XGBoost, random forest, approximate k-nearest neighbor, elastic net, support vector machines and a separate real-world motivated optimisation problem. We open source the benchmarks to foster future research on ordered transfer HPO.

翻译：我们提出有序迁移超参数优化（OTHPO），这是一种针对超参数优化（HPO）的迁移学习变体，其中任务遵循序贯顺序。与最先进的迁移HPO不同，本方法假设每个任务与其紧邻的前序任务相关性最高。这符合许多实际部署场景——随着数据积累需重新调整超参数，例如在电影推荐系统中，随着新电影和评分的增加对超参数序列进行调优。我们给出形式化定义，阐述其与相关问题的差异，并提出一种超越现有最优迁移HPO方法的基础OTHPO方法。通过十个基准实验，我们实证证明了考虑序贯顺序的重要性。这些基准实验模拟数据逐步累积的环境，涵盖XGBoost、随机森林、近似K近邻、弹性网络、支持向量机以及一个独立的现实世界优化问题。我们开源所有基准测试，以促进有序迁移HPO的未来研究。