We study the constrained variant of the \emph{multi-armed bandit} (MAB) problem, in which the learner aims not only at minimizing the total loss incurred during the learning dynamic, but also at controlling the violation of multiple \emph{unknown} constraints, under both \emph{full} and \emph{bandit feedback}. We consider a non-stationary environment that subsumes both stochastic and adversarial models and where, at each round, both losses and constraints are drawn from distributions that may change arbitrarily over time. In such a setting, it is provably not possible to guarantee both sublinear regret and sublinear violation. Accordingly, prior work has mainly focused either on settings with stochastic constraints or on relaxing the benchmark with fully adversarial constraints (\emph{e.g.}, via competitive ratios with respect to the optimum). We provide the first algorithms that achieve optimal rates of regret and \emph{positive} constraint violation when the constraints are stochastic while the losses may vary arbitrarily, and that simultaneously yield guarantees that degrade smoothly with the degree of adversariality of the constraints. Specifically, under \emph{full feedback} we propose an algorithm attaining $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ regret and $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ {positive} violation, where $C$ quantifies the amount of non-stationarity in the constraints. We then show how to extend these guarantees when only bandit feedback is available for the losses. Finally, when \emph{bandit feedback} is available for the constraints, we design an algorithm achieving $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ {positive} violation and $\widetilde{\mathcal{O}}(\sqrt{T}+C\sqrt{T})$ regret.

翻译：我们研究了\emph{多臂老虎机}（MAB）问题的约束变体，其中学习者的目标不仅在于最小化学习动态过程中产生的总损失，还在于在\emph{完全反馈}和\emph{老虎机反馈}两种情况下，控制对多个\emph{未知}约束的违反程度。我们考虑一个非平稳环境，该环境涵盖了随机和对抗模型，其中每轮次的损失和约束均从可能随时间任意变化的分布中抽取。在此类设置下，理论上无法同时保证次线性遗憾和次线性违反。因此，先前研究主要集中于具有随机约束的设置，或通过完全对抗性约束下的基准放松（\emph{例如}，相对于最优解的竞争比）来解决问题。我们提出了首批算法，这些算法在约束为随机性而损失可能任意变化时，实现了最优的遗憾率和\emph{正向}约束违反率，并且同时能保证其性能随约束对抗性程度的增加而平滑下降。具体而言，在\emph{完全反馈}下，我们提出一种算法，达到 $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ 的遗憾和 $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ 的{正向}违反，其中 $C$ 量化了约束中的非平稳性程度。随后，我们展示了当仅能获得损失的老虎机反馈时，如何扩展这些保证。最后，当约束可获得\emph{老虎机反馈}时，我们设计了一种算法，实现 $\widetilde{\mathcal{O}}(\sqrt{T}+C)$ 的{正向}违反和 $\widetilde{\mathcal{O}}(\sqrt{T}+C\sqrt{T})$ 的遗憾。