This paper considers a stochastic multi-armed bandit (MAB) problem with dual objectives: (i) quick identification and commitment to the optimal arm, and (ii) reward maximization throughout a sequence of $T$ consecutive rounds. Though each objective has been individually well-studied, i.e., best arm identification for (i) and regret minimization for (ii), the simultaneous realization of both objectives remains an open problem, despite its practical importance. This paper introduces \emph{Regret Optimal Best Arm Identification} (ROBAI) which aims to achieve these dual objectives. To solve ROBAI with both pre-determined stopping time and adaptive stopping time requirements, we present the $\mathsf{EOCP}$ algorithm and its variants respectively, which not only achieve asymptotic optimal regret in both Gaussian and general bandits, but also commit to the optimal arm in $\mathcal{O}(\log T)$ rounds with pre-determined stopping time and $\mathcal{O}(\log^2 T)$ rounds with adaptive stopping time. We further characterize lower bounds on the commitment time (equivalent to sample complexity) of ROBAI, showing that $\mathsf{EOCP}$ and its variants are sample optimal with pre-determined stopping time, and almost sample optimal with adaptive stopping time. Numerical results confirm our theoretical analysis and reveal an interesting ``over-exploration'' phenomenon carried by classic $\mathsf{UCB}$ algorithms, such that $\mathsf{EOCP}$ has smaller regret even though it stops exploration much earlier than $\mathsf{UCB}$ ($\mathcal{O}(\log T)$ versus $\mathcal{O}(T)$), which suggests over-exploration is unnecessary and potentially harmful to system performance.

翻译：本文考虑一个具有双重目标的随机多臂赌博机（MAB）问题：（i）快速识别并锁定最优臂，（ii）在$T$个连续回合序列中最大化累积奖励。尽管每个目标均已得到单独深入研究——即（i）的最佳臂识别与（ii）的遗憾最小化——但这两个目标的同时实现仍是一个开放问题，尽管其具有重要的实践意义。本文提出**憾最优最佳臂识别**（ROBAI），旨在实现上述双重目标。针对具有预定停止时间和自适应停止时间需求的ROBAI问题，我们分别提出$\mathsf{EOCP}$算法及其变体，这些算法不仅在高斯和一般赌博机场景下实现了渐近最优遗憾，还能在预定停止时间下于$\mathcal{O}(\log T)$轮内锁定最优臂，在自适应停止时间下于$\mathcal{O}(\log^2 T)$轮内锁定最优臂。我们进一步刻画了ROBAI在锁定时间（等价于样本复杂度）上的下界，证明$\mathsf{EOCP}$及其变体在预定停止时间下达到样本最优，在自适应停止时间下接近样本最优。数值结果验证了我们的理论分析，并揭示了经典$\mathsf{UCB}$算法存在的有趣“过度探索”现象：$\mathsf{EOCP}$虽比$\mathsf{UCB}$更早停止探索（$\mathcal{O}(\log T)$对比$\mathcal{O}(T)$），但其遗憾值更小，表明过度探索不必要且可能损害系统性能。