Given a set of arms $\mathcal{Z}\subset \mathbb{R}^d$ and an unknown parameter vector $\theta_\ast\in\mathbb{R}^d$, the pure exploration linear bandit problem aims to return $\arg\max_{z\in \mathcal{Z}} z^{\top}\theta_{\ast}$, with high probability through noisy measurements of $x^{\top}\theta_{\ast}$ with $x\in \mathcal{X}\subset \mathbb{R}^d$. Existing (asymptotically) optimal methods require either a) potentially costly projections for each arm $z\in \mathcal{Z}$ or b) explicitly maintaining a subset of $\mathcal{Z}$ under consideration at each time. This complexity is at odds with the popular and simple Thompson Sampling algorithm for regret minimization, which just requires access to a posterior sampling and argmax oracle, and does not need to enumerate $\mathcal{Z}$ at any point. Unfortunately, Thompson sampling is known to be sub-optimal for pure exploration. In this work, we pose a natural question: is there an algorithm that can explore optimally and only needs the same computational primitives as Thompson Sampling? We answer the question in the affirmative. We provide an algorithm that leverages only sampling and argmax oracles and achieves an exponential convergence rate, with the exponent being the optimal among all possible allocations asymptotically. In addition, we show that our algorithm can be easily implemented and performs as well empirically as existing asymptotically optimal methods.

翻译：给定一组臂 $\mathcal{Z}\subset \mathbb{R}^d$ 和一个未知参数向量 $\theta_\ast\in\mathbb{R}^d$，纯探索线性臂问题旨在通过带有噪声的对 $x^{\top}\theta_{\ast}$（其中 $x\in \mathcal{X}\subset \mathbb{R}^d$）的测量，以高概率返回 $\arg\max_{z\in \mathcal{Z}} z^{\top}\theta_{\ast}$。现有的（渐近）最优方法要么 a) 需要对每个臂 $z\in \mathcal{Z}$ 进行可能成本高昂的投影，要么 b) 在每个时刻显式维护一个待考虑的 $\mathcal{Z}$ 子集。这种复杂度与用于遗憾最小化的流行且简单的汤普森采样算法相矛盾，后者仅需访问后验采样和 argmax 预言，且无需在任何时刻枚举 $\mathcal{Z}$。遗憾的是，已知汤普森采样对于纯探索而言是次优的。本文提出一个自然问题：是否存在一种能够进行最优探索、且仅需与汤普森采样相同计算原语的算法？我们对此问题给出肯定回答。我们提供了一种仅利用采样和 argmax 预言的算法，实现了指数收敛速率，且该指数在渐近意义下是所有可能分配中的最优值。此外，我们证明该算法易于实现，并在实证中表现得与现有渐近最优方法一样出色。