We address the problem of identifying the optimal policy with a fixed confidence level in a multi-armed bandit setup, when \emph{the arms are subject to linear constraints}. Unlike the standard best-arm identification problem which is well studied, the optimal policy in this case may not be deterministic and could mix between several arms. This changes the geometry of the problem which we characterize via an information-theoretic lower bound. We introduce two asymptotically optimal algorithms for this setting, one based on the Track-and-Stop method and the other based on a game-theoretic approach. Both these algorithms try to track an optimal allocation based on the lower bound and computed by a weighted projection onto the boundary of a normal cone. Finally, we provide empirical results that validate our bounds and visualize how constraints change the hardness of the problem.

翻译：我们研究了在多臂老虎机框架下，当臂受到线性约束时，以固定置信度识别最优策略的问题。与已有充分研究的标准最优臂识别问题不同，此情形下的最优策略可能非确定性，且需在多个臂之间进行混合。这改变了问题的几何结构，我们通过信息论下界对其进行刻画。我们针对该设定提出了两种渐近最优算法：一种基于追踪-停止方法，另一种基于博弈论方法。这两种算法均尝试根据下界跟踪最优分配，并通过加权投影至法锥边界来求解。最后，我们通过实验结果验证了理论界，并展示了约束如何改变问题的求解难度。