The knowledge gradient is a popular acquisition function in Bayesian optimization (BO) for optimizing black-box objectives with noisy function evaluations. Many practical settings, however, allow only pairwise comparison queries, yielding a preferential BO problem where direct function evaluations are unavailable. Extending the knowledge gradient to preferential BO is hindered by its computational challenge. At its core, the look-ahead step in the preferential setting requires computing a non-Gaussian posterior, which was previously considered intractable. In this paper, we address this challenge by deriving an exact and analytical knowledge gradient for preferential BO. We show that the exact knowledge gradient performs strongly on a suite of benchmark problems, often outperforming existing acquisition functions. In addition, we also present a case study illustrating the limitation of the knowledge gradient in certain scenarios.

翻译：知识梯度是贝叶斯优化中一种常用的采集函数，用于在含噪声函数评估条件下优化黑箱目标函数。然而，许多实际场景仅允许进行成对比较查询，从而形成了直接函数评估不可用的偏好贝叶斯优化问题。将知识梯度扩展至偏好贝叶斯优化领域一直受限于其计算复杂性。其核心困难在于：偏好场景中的前瞻步骤需要计算非高斯后验分布，此前该问题被认为无法解析处理。本文通过推导偏好贝叶斯优化的精确解析知识梯度来解决这一挑战。我们证明该精确知识梯度在基准测试集上表现优异，通常优于现有采集函数。此外，我们还通过案例研究揭示了知识梯度在特定场景下的局限性。