We study the stochastic Budgeted Multi-Armed Bandit (MAB) problem, where a player chooses from $K$ arms with unknown expected rewards and costs. The goal is to maximize the total reward under a budget constraint. A player thus seeks to choose the arm with the highest reward-cost ratio as often as possible. Current state-of-the-art policies for this problem have several issues, which we illustrate. To overcome them, we propose a new upper confidence bound (UCB) sampling policy, $\omega$-UCB, that uses asymmetric confidence intervals. These intervals scale with the distance between the sample mean and the bounds of a random variable, yielding a more accurate and tight estimation of the reward-cost ratio compared to our competitors. We show that our approach has logarithmic regret and consistently outperforms existing policies in synthetic and real settings.

翻译：我们研究随机预算型多臂老虎机（MAB）问题，其中玩家在$K$个具有未知期望收益和成本的臂中进行选择。目标是在预算约束下最大化总收益，因此玩家需尽可能频繁地选择收益-成本比最高的臂。针对该问题，现有最优策略存在若干缺陷，本文对此进行了说明。为克服这些缺陷，我们提出一种新的上置信界（UCB）采样策略——$\omega$-UCB，该策略采用非对称置信区间。此类区间随样本均值与随机变量界限之间的距离动态缩放，相比现有方法，能更精确、更紧凑地估计收益-成本比。我们证明，该方法的遗憾值呈对数增长，并在合成与真实环境中始终优于现有策略。