Bayesian optimization (BO) with Gaussian processes (GP) has become an indispensable algorithm for black box optimization problems. Not without a dash of irony, BO is often considered a black box itself, lacking ways to provide reasons as to why certain parameters are proposed to be evaluated. This is particularly relevant in human-in-the-loop applications of BO, such as in robotics. We address this issue by proposing ShapleyBO, a framework for interpreting BO's proposals by game-theoretic Shapley values.They quantify each parameter's contribution to BO's acquisition function. Exploiting the linearity of Shapley values, we are further able to identify how strongly each parameter drives BO's exploration and exploitation for additive acquisition functions like the confidence bound. We also show that ShapleyBO can disentangle the contributions to exploration into those that explore aleatoric and epistemic uncertainty. Moreover, our method gives rise to a ShapleyBO-assisted human machine interface (HMI), allowing users to interfere with BO in case proposals do not align with human reasoning. We demonstrate this HMI's benefits for the use case of personalizing wearable robotic devices (assistive back exosuits) by human-in-the-loop BO. Results suggest human-BO teams with access to ShapleyBO can achieve lower regret than teams without.

翻译：基于高斯过程的贝叶斯优化已成为解决黑箱优化问题不可或缺的算法。颇具讽刺意味的是，贝叶斯优化自身往往被视为一个黑箱，无法解释为何提议评估特定参数。这一点在人与回路协同的贝叶斯优化应用中尤为重要，例如机器人领域。我们提出ShapleyBO框架来解决此问题，该框架通过博弈论中的沙普利值解读贝叶斯优化的提议。沙普利值可量化每个参数对贝叶斯优化采集函数的贡献程度。利用沙普利值的线性可加性，我们进一步能够识别每个参数在置信边界等可加性采集函数中如何影响贝叶斯优化的探索与利用平衡。研究还表明，ShapleyBO可分离出探索行为中针对偶然不确定性与认知不确定性的贡献。此外，该方法催生了ShapleyBO辅助人机界面，允许用户在提议与人类推理不一致时干预贝叶斯优化过程。我们通过人体回路贝叶斯优化个性化可穿戴机器人设备（助力背心外骨骼）的案例，展示了该人机界面的优势。结果表明，配备ShapleyBO的人机协作团队在优化后悔值上显著优于未使用该工具的团队。