A key component of automated algorithm selection and configuration, which in most cases are performed using supervised machine learning (ML) methods is a good-performing predictive model. The predictive model uses the feature representation of a set of problem instances as input data and predicts the algorithm performance achieved on them. Common machine learning models struggle to make predictions for instances with feature representations not covered by the training data, resulting in poor generalization to unseen problems. In this study, we propose a workflow to estimate the generalizability of a predictive model for algorithm performance, trained on one benchmark suite to another. The workflow has been tested by training predictive models across benchmark suites and the results show that generalizability patterns in the landscape feature space are reflected in the performance space.

翻译：自动化算法选择与配置的关键组成部分（多数情况下使用监督式机器学习方法实现）是一个性能良好的预测模型。该模型以问题实例集的特征表示作为输入数据，预测算法在这些实例上所能达到的性能。常见的机器学习模型难以对训练数据未覆盖的特征表示实例进行有效预测，导致其对未见问题的泛化能力不足。本研究提出了一套工作流，用于评估针对算法性能的预测模型从某一基准测试集迁移至另一基准测试集时的泛化能力。通过跨基准测试集训练预测模型进行测试，结果表明：景观特征空间中的泛化模式会反映在性能空间中。