Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in the final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nystrom approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve state-of-the-art performance.
翻译:贝叶斯优化(BO)是一种样本高效的优化算法,广泛应用于各类场景。在部分具有挑战性的贝叶斯优化任务中,由于加工误差、执行噪声或上下文变化等优化过程中不可避免的随机性,输入会产生不确定性。这种不确定性导致评估前实际输入偏离预期值,最终造成优化结果性能显著波动。本文提出一种新型稳健贝叶斯优化算法AIRBO,该算法能够有效识别在任意输入不确定性下均表现稳定的稳健最优解。该方法通过将最大均值差异(MMD)引入高斯过程来直接建模任意分布的输入不确定性,并进一步采用Nyström近似加速后验推理。我们建立了基于MMD估计误差的严格理论遗憾界,在合成函数和真实问题上的大量实验表明,该方法能够处理多种输入不确定性并达到当前最优性能。