We introduce the E$^4$ algorithm for the batched linear bandit problem, incorporating an Explore-Estimate-Eliminate-Exploit framework. With a proper choice of exploration rate, we prove E$^4$ achieves the finite-time minimax optimal regret with only $O(\log\log T)$ batches, and the asymptotically optimal regret with only $3$ batches as $T\rightarrow\infty$, where $T$ is the time horizon. We further prove a lower bound on the batch complexity of linear contextual bandits showing that any asymptotically optimal algorithm must require at least $3$ batches in expectation as $T\rightarrow\infty$, which indicates E$^4$ achieves the asymptotic optimality in regret and batch complexity simultaneously. To the best of our knowledge, E$^4$ is the first algorithm for linear bandits that simultaneously achieves the minimax and asymptotic optimality in regret with the corresponding optimal batch complexities. In addition, we show that with another choice of exploration rate E$^4$ achieves an instance-dependent regret bound requiring at most $O(\log T)$ batches, and maintains the minimax optimality and asymptotic optimality. We conduct thorough experiments to evaluate our algorithm on randomly generated instances and the challenging \textit{End of Optimism} instances \citep{lattimore2017end} which were shown to be hard to learn for optimism based algorithms. Empirical results show that E$^4$ consistently outperforms baseline algorithms with respect to regret minimization, batch complexity, and computational efficiency.

翻译：我们针对批次线性赌博机问题提出了E$^4$算法，该算法融合了探索-估计-消除-利用（Explore-Estimate-Eliminate-Exploit）框架。通过合理选择探索率，我们证明E$^4$仅需$O(\log\log T)$个批次即可实现有限时间极小化最优遗憾，并且在$T\rightarrow\infty$时仅需$3$个批次即可达到渐近最优遗憾，其中$T$为时间范围。我们进一步证明了线性上下文赌博机的批次复杂度下界：当$T\rightarrow\infty$时，任何渐近最优算法在期望意义下至少需要$3$个批次，这表明E$^4$在遗憾和批次复杂度上同时达到了渐近最优性。据我们所知，E$^4$是首个在批次线性赌博机中同时实现遗憾的极小化最优性、渐近最优性以及对应最优批次复杂度的算法。此外，我们证明通过选择另一种探索率，E$^4$可实现基于实例的遗憾界（最多需要$O(\log T)$个批次），同时保持极小化最优性和渐近最优性。我们通过随机生成的实例以及具有挑战性的\textit{End of Optimism}实例（该实例被证明对基于乐观的算法难以学习，参见\citep{lattimore2017end}）进行了详尽的实验评估。实证结果表明，E$^4$在遗憾最小化、批次复杂度和计算效率方面始终优于基线算法。