We study the tail behavior of regret in stochastic multi-armed bandits for algorithms that are asymptotically optimal in expectation. While minimizing expected regret is the classical objective, recent work shows that even such algorithms can exhibit heavy regret tails, incurring large regret with non-negligible probability. Existing sharp characterizations of regret tails are largely restricted to parametric settings, such as single-parameter exponential families. In this work, we extend the $\KLinf$-UCB algorithm of to a broad nonparametric class of reward distributions satisfying mild assumptions, and establish its asymptotic optimality in expectation. We then analyze the tail behavior of its regret and derive a novel upper bound on the regret tail probability. As special cases, our results recover regret-tail guarantees for both bounded-support and heavy-tailed (moment-bounded) bandit models. Moreover, for the special case of finitely-supported reward distributions, our upper bound matches the known lower bound exactly. Our results thus provide a unified and tight characterization of regret tails for asymptotically optimal KL-based UCB algorithms, going beyond parametric models.

翻译：我们研究随机多臂老虎机中遗憾的尾部行为，针对渐近期望最优的算法。虽然最小化期望遗憾是经典目标，但近期研究表明，即便是此类算法也可能表现出严重的遗憾尾部现象，即非可忽略概率下产生较大遗憾。现有关于遗憾尾部的精确刻画主要局限于参数化设定，例如单参数指数族。本研究将$\KLinf$-UCB算法拓展至满足温和假设的广泛非参数奖励分布类，并证明其渐近期望最优性。随后分析其遗憾的尾部行为，推导出遗憾尾部概率的新上界。作为特例，我们的结果同时涵盖了有界支撑和重尾（矩有界）bandit模型的遗憾尾部保证。此外，对于有限支撑奖励分布这一特殊情形，该上界与已知下界精确匹配。因此，我们的结果为渐近最优的基于KL的UCB算法提供了统一且紧致的遗憾尾部刻画，突破了参数化模型的限制。