Heuristic optimisation algorithms explore the search space by sampling solutions, evaluating their fitness, and biasing the search in the direction of promising solutions. However, in many cases, this fitness function involves executing expensive computational calculations, drastically reducing the reasonable number of evaluations. In this context, surrogate models have emerged as an excellent alternative to alleviate these computational problems. This paper addresses the formulation of surrogate problems as both regression models that approximate fitness (surface surrogate models) and a novel way to connect classification models (pairwise surrogate models). The pairwise approach can be directly exploited by some algorithms, such as Differential Evolution, in which the fitness value is not actually needed to drive the search, and it is sufficient to know whether a solution is better than another one or not. Based on these modelling approaches, we have conducted a multidimensional analysis of surrogate models under different configurations: different machine learning algorithms (regularised regression, neural networks, decision trees, boosting methods, and random forests), different surrogate strategies (encouraging diversity or relaxing prediction thresholds), and compare them for both surface and pairwise surrogate models. The experimental part of the article includes the benchmark problems already proposed for the SOCO2011 competition in continuous optimisation and a simulation problem included in the recent GECCO2021 Industrial Challenge. This paper shows that the performance of the overall search, when using online machine learning-based surrogate models, depends not only on the accuracy of the predictive model but also on both the kind of bias towards positive or negative cases and how the optimisation uses those predictions to decide whether to execute the actual fitness function.

翻译：启发式优化算法通过采样解、评估其适应度，并向有前景的解方向偏置搜索来探索搜索空间。然而，在许多情况下，该适应度函数涉及执行昂贵的计算过程，从而大幅减少了合理的评估次数。在此背景下，代理模型已成为缓解这些计算问题的优秀替代方案。本文探讨了将代理问题同时表述为近似适应度的回归模型（曲面代理模型）以及一种连接分类模型的新方法（成对代理模型）。成对方法可直接被某些算法（如差分进化算法）利用，在这类算法中驱动搜索实际上并不需要适应度值，仅需知道一个解是否优于另一个解。基于这些建模方法，我们对不同配置下的代理模型进行了多维分析：不同的机器学习算法（正则化回归、神经网络、决策树、提升方法和随机森林）、不同的代理策略（鼓励多样性或放宽预测阈值），并针对曲面代理模型和成对代理模型进行了比较。本文的实验部分包含已为SOCO2011连续优化竞赛提出的基准问题，以及近期GECCO2021工业挑战赛中的一个仿真问题。本文表明，当使用基于在线机器学习的代理模型时，整体搜索性能不仅取决于预测模型的准确性，还取决于对正负例的偏置类型以及优化算法如何利用这些预测来决定是否执行实际适应度函数。