Algorithms for offline bandits must optimize decisions in uncertain environments using only offline data. A compelling and increasingly popular objective in offline bandits is to learn a policy which achieves low Bayesian regret with high confidence. An appealing approach to this problem, inspired by recent offline reinforcement learning results, is to maximize a form of lower confidence bound (LCB). This paper proposes a new approach that directly minimizes upper bounds on Bayesian regret using efficient conic optimization solvers. Our bounds build on connections among Bayesian regret, Value-at-Risk (VaR), and chance-constrained optimization. Compared to prior work, our algorithm attains superior theoretical offline regret bounds and better results in numerical simulations. Finally, we provide some evidence that popular LCB-style algorithms may be unsuitable for minimizing Bayesian regret in offline bandits.

翻译：离线强盗算法必须仅利用离线数据在不确定环境中优化决策。在离线强盗问题中，一个具有吸引力且日益流行的目标是学习一个能够以高置信度实现低贝叶斯遗憾的策略。受近期离线强化学习成果启发，解决该问题的一种有效方法是最大化某种形式的置信下界（LCB）。本文提出了一种新方法，利用高效的锥优化求解器直接最小化贝叶斯遗憾的上界。我们的界限建立在贝叶斯遗憾、风险价值（VaR）与机会约束优化之间的关联之上。与先前工作相比，我们的算法在理论上获得了更优的离线遗憾界，并在数值模拟中展现出更佳结果。最后，我们提供证据表明，流行的LCB类算法可能不适用于最小化离线强盗问题中的贝叶斯遗憾。