Multi-fidelity machine learning methods address the accuracy-efficiency trade-off by integrating scarce, resource-intensive high-fidelity data with abundant but less accurate low-fidelity data. We propose a practical multi-fidelity strategy for problems spanning low- and high-dimensional domains, integrating a non-probabilistic regression model for the low-fidelity with a Bayesian model for the high-fidelity. The models are trained in a staggered scheme, where the low-fidelity model is transfer-learned to the high-fidelity data and a Bayesian model is trained to learn the residual between the data and the transfer-learned model. This three-model strategy -- deterministic low-fidelity, transfer-learning, and Bayesian residual -- leads to a prediction that includes uncertainty quantification for noisy and noiseless multi-fidelity data. The strategy is general and unifies the topic, highlighting the expressivity trade-off between the transfer-learning and Bayesian models (a complex transfer-learning model leads to a simpler Bayesian model, and vice versa). We propose modeling choices for two scenarios, and argue in favor of using a linear transfer-learning model that fuses 1) kernel ridge regression for low-fidelity with Gaussian processes for high-fidelity; or 2) deep neural network for low-fidelity with a Bayesian neural network for high-fidelity. We demonstrate the effectiveness and efficiency of the proposed strategies and contrast them with the state-of-the-art based on various numerical examples and two engineering problems. The results indicate that the proposed approach achieves comparable performance in both mean and uncertainty estimation while significantly reducing training time for machine learning modeling in data-scarce scenarios. Moreover, in data-rich settings, it outperforms other multi-fidelity architectures by effectively mitigating overfitting.

翻译：多保真度机器学习方法通过整合稀缺且资源密集的高保真度数据与丰富但精度较低的低保真度数据，以应对精度与效率之间的权衡。本文针对涵盖低维与高维领域的问题，提出一种实用的多保真度策略，将用于低保真度的非概率回归模型与用于高保真度的贝叶斯模型相结合。模型采用交错训练方案：首先对低保真度模型进行迁移学习以适应高保真度数据，随后训练贝叶斯模型以学习数据与迁移学习模型之间的残差。这种由确定性低保真度模型、迁移学习模型和贝叶斯残差模型构成的三模型策略，能够为含噪声及无噪声的多保真度数据提供包含不确定性量化的预测。该策略具有普适性，统一了该研究主题，并揭示了迁移学习模型与贝叶斯模型之间的表达能力权衡（复杂的迁移学习模型会导致贝叶斯模型简化，反之亦然）。针对两种典型场景，我们提出相应的建模方案：1）将用于低保真度的核岭回归与用于高保真度的高斯过程相结合；或2）将用于低保真度的深度神经网络与用于高保真度的贝叶斯神经网络相结合，并主张采用线性迁移学习模型进行融合。通过多种数值算例和两个工程问题的验证，我们证明了所提策略的有效性与高效性，并与现有先进方法进行了对比。结果表明，在数据稀缺场景下，所提方法在均值预测和不确定性估计方面均能达到可比性能，同时显著缩短了机器学习建模的训练时间。此外，在数据充足场景中，该方法通过有效缓解过拟合问题，超越了其他多保真度架构。