We give a near-optimal sample-pass trade-off for pure exploration in multi-armed bandits (MABs) via multi-pass streaming algorithms: any streaming algorithm with sublinear memory that uses the optimal sample complexity of $O(\frac{n}{\Delta^2})$ requires $\Omega(\frac{\log{(1/\Delta)}}{\log\log{(1/\Delta)}})$ passes. Here, $n$ is the number of arms and $\Delta$ is the reward gap between the best and the second-best arms. Our result matches the $O(\log(\frac{1}{\Delta}))$-pass algorithm of Jin et al. [ICML'21] (up to lower order terms) that only uses $O(1)$ memory and answers an open question posed by Assadi and Wang [STOC'20].
翻译:我们为多臂老虎机(MAB)中的纯探索问题给出了一个几乎最优的样本-遍数权衡:任何使用最优样本复杂度$O(\frac{n}{\Delta^2})$、内存亚线性的流算法,至少需要$\Omega(\frac{\log{(1/\Delta)}}{\log\log{(1/\Delta)}})$遍。这里,$n$是臂数,$\Delta$是最优臂与次优臂之间的奖励差距。我们的结果匹配了Jin等人[ICML'21]仅使用$O(1)$内存的$O(\log(\frac{1}{\Delta}))$遍算法(直至低阶项),并回答了Assadi与Wang[STOC'20]提出的开放问题。