Surrogate modelling techniques have seen growing attention in recent years when applied to both modelling and optimisation of industrial design problems. These techniques are highly relevant when assessing the performance of a particular design carries a high cost, as the overall cost can be mitigated via the construction of a model to be queried in lieu of the available high-cost source. The construction of these models can sometimes employ other sources of information which are both cheaper and less accurate. The existence of these sources however poses the question of which sources should be used when constructing a model. Recent studies have attempted to characterise harmful data sources to guide practitioners in choosing when to ignore a certain source. These studies have done so in a synthetic setting, characterising sources using a large amount of data that is not available in practice. Some of these studies have also been shown to potentially suffer from bias in the benchmarks used in the analysis. In this study, we present a characterisation of harmful low-fidelity sources using only the limited data available to train a surrogate model. We employ recently developed benchmark filtering techniques to conduct a bias-free assessment, providing objectively varied benchmark suites of different sizes for future research. Analysing one of these benchmark suites with the technique known as Instance Space Analysis, we provide an intuitive visualisation of when a low-fidelity source should be used and use this analysis to provide guidelines that can be used in an applied industrial setting.

翻译：代理模型技术近年来在工业设计问题的建模与优化中受到日益关注。当评估特定设计的性能成本高昂时，这些技术尤为重要，因为可以通过构建代理模型来替代高成本数据源进行查询，从而降低总成本。在构建这类模型时，有时会采用其他成本较低但精度也较低的数据源。然而，这些数据源的存在引发了一个问题：在构建模型时应选用哪些数据源？近期研究试图表征有害数据源，以指导实践者在特定情况下选择忽略某些数据源。这些研究在合成环境中进行，利用实践中无法获取的大量数据来表征数据源。部分研究还被发现可能因分析中所用基准存在偏差而受到影响。在本研究中，我们仅利用训练代理模型时可获得的有限数据，对有害低保真度数据源进行了特征化。我们采用近期开发的基准过滤技术进行无偏差评估，为未来研究提供了不同规模且客观多样的基准套件。通过一种称为实例空间分析的技术对其中一个基准套件进行分析，我们直观展示了何时应使用低保真度数据源，并基于此分析提出可在工业应用环境中使用的指导原则。