In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by a stringent budget constraint on the available resources, which are consumed in a random amount by each action, and a stochastic feasibility constraint that may impose important operational limitations on decision-making. In this work, we consider a general model to address such problems, where each action returns a random reward, cost, and penalty from an unknown joint distribution, and the decision-maker aims to maximize the total reward under a budget constraint $B$ on the total cost and a stochastic constraint on the time-average penalty. We propose a novel low-complexity algorithm based on Lyapunov optimization methodology, named ${\tt LyOn}$, and prove that for $K$ arms it achieves $O(\sqrt{K B\log B})$ regret and zero constraint-violation when $B$ is sufficiently large. The low computational cost and sharp performance bounds of ${\tt LyOn}$ suggest that Lyapunov-based algorithm design methodology can be effective in solving constrained bandit optimization problems.

翻译：在各种应用中,包括网上广告、合同雇用和无线日程安排,控制员受到以下因素的限制:对现有资源的严格预算限制,每项行动都随机消耗大量资源,以及可能给决策造成重大业务限制的随机可行性限制。在这项工作中,我们考虑一种解决这类问题的一般模式,即每次行动都带来随机报酬、费用和由未知的联合分配所施加的惩罚,决策者的目标是在预算限制下,在总成本的B美元和对时间平均罚款的随机限制下,最大限度地获得全部报酬。我们提议一种基于Lyapunov优化方法的新的低复杂性算法,名为$&t LyOn},并证明如果$B足够大,对$(sqrt{K B\log B})产生遗憾和零限制。低计算成本和急性性性能约束为$@tt LyOn}表明,基于Lyapunov的算法设计方法能够有效地解决限制的波段优化问题。