The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. (2020), which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in $\phi$-divergences, which subsumes many popular choices, such as the $\chi^2$-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results.

翻译：鲁棒性研究在数据驱动环境中备受关注，因为许多系统在此类场景下不可避免地面临不确定性。贝叶斯优化（BO）便是其中典型实例——其不确定性具有多面性，但相关研究方向的工作却十分有限。特别地，Kirschner等人（2020）的研究通过从分布鲁棒优化（DRO）视角重构贝叶斯优化问题，架起了现有分布鲁棒优化文献的桥梁。尽管这项研究具有开创性，但不可否认存在诸多实践缺陷（如有限上下文假设），遗留了核心问题：能否设计出可计算高效的算法来解决这一DRO-BO问题？本研究在高度普适性下解决该问题，考虑针对φ-散度下数据偏移的鲁棒性。φ-散度涵盖多种常用形式，如χ²散度、全变差散度及现有KL散度。我们证明该场景下的DRO-BO问题等价于一个有限维优化问题——即使在连续上下文设定中，该问题亦能轻松实现并伴有可证明的次线性遗憾界。实验结果表明，本方法在性能上超越现有方法，验证了理论推导的正确性。