In this paper, we study the bandits with knapsacks (BwK) problem and develop a primal-dual based algorithm that achieves a problem-dependent logarithmic regret bound. The BwK problem extends the multi-arm bandit (MAB) problem to model the resource consumption associated with playing each arm, and the existing BwK literature has been mainly focused on deriving asymptotically optimal distribution-free regret bounds. We first study the primal and dual linear programs underlying the BwK problem. From this primal-dual perspective, we discover symmetry between arms and knapsacks, and then propose a new notion of sub-optimality measure for the BwK problem. The sub-optimality measure highlights the important role of knapsacks in determining algorithm regret and inspires the design of our two-phase algorithm. In the first phase, the algorithm identifies the optimal arms and the binding knapsacks, and in the second phase, it exhausts the binding knapsacks via playing the optimal arms through an adaptive procedure. Our regret upper bound involves the proposed sub-optimality measure and it has a logarithmic dependence on length of horizon $T$ and a polynomial dependence on $m$ (the numbers of arms) and $d$ (the number of knapsacks). To the best of our knowledge, this is the first problem-dependent logarithmic regret bound for solving the general BwK problem.

翻译：在本文中,我们用 knapsack (BwK) 问题来研究土匪, 并开发一个基于原始的双向算法, 从而实现一个基于问题的对数质质折数。 BwK 问题扩大了多臂匪(MAB) 问题的范围, 以模拟与玩弄每只手臂相关的资源消耗, 而现有的 BwK 文献则主要侧重于得出非现最佳分配- 无悔恨界限。 我们首先研究 BwK 问题背后的原始和双重线性程序。 从这个原始的角度看, 我们发现武器和 knaps 之间的对称, 然后为 BwK 问题提出了一个新概念。 BwK 问题次优化度测量突出了 knaps 在确定算法遗憾和启发我们两阶段算法的设计中的重要作用。 在第一阶段, 算法确定了最佳的手臂和固定的 knapsack 问题, 在第二个阶段, 我们通过最优化的 $ 美元 的对数的对数的对数的对数调程序, 我们的对数的对数的对数排序进行拟议的亚度测量。