The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. (2020), which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in $\varphi$-divergences, which subsumes many popular choices, such as the $\chi^2$-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results.

翻译：鲁棒性研究因数据驱动场景中诸多系统面临不确定性的必然性而备受关注。一个典型实例是贝叶斯优化（Bayesian Optimization, BO），其不确定性具有多面性，但目前仅有少量工作致力于这一方向。特别是Kirschner等人（2020）的工作，通过从分布鲁棒优化（Distributionally Robust Optimization, DRO）视角重构BO问题，架起了现有DRO文献的桥梁。尽管这一工作具有开创性，但不可否认其存在实际应用中的诸多局限性，如有限环境假设，从而遗留了核心问题：能否设计出计算可处理的算法来解决此DRO-BO问题？本文在高度一般性下解决了该问题，通过考虑基于$\varphi$-散度的数据偏移鲁棒性，该框架涵盖了多种常见选择，如$\chi^2$-散度、全变差以及已有的Kullback-Leibler（KL）散度。我们证明了该设定下的DRO-BO问题等价于一个有限维优化问题，即使在连续环境设定下，也能以可证明的次线性遗憾界轻松实现。实验结果表明，我们的方法超越了现有方法，验证了理论结果。