We introduce the Thresholding Monte Carlo Tree Search problem, in which, given a tree $\mathcal{T}$ and a threshold $θ$, a player must answer whether the root node value of $\mathcal{T}$ is at least $θ$ or not. In the given tree, `MAX' or `MIN' is labeled on each internal node, and the value of a `MAX'-labeled (`MIN'-labeled) internal node is the maximum (minimum) of its child values. The value of a leaf node is the mean reward of an unknown distribution, from which the player can sample rewards. For this problem, we develop a $δ$-correct sequential sampling algorithm based on the Track-and-Stop strategy that has asymptotically optimal sample complexity. We show that a ratio-based modification of the D-Tracking arm-pulling strategy leads to a substantial improvement in empirical sample complexity, as well as reducing the per-round computational cost from linear to logarithmic in the number of arms.

翻译：我们引入了阈值蒙特卡洛树搜索问题，其中给定一棵树 $\mathcal{T}$ 和一个阈值 $θ$，玩家必须判断 $\mathcal{T}$ 的根节点值是否至少为 $θ$。在给定的树中，每个内部节点被标记为“MAX”或“MIN”，标记为“MAX”（“MIN”）的内部节点的值是其子节点值的最大值（最小值）。叶节点的值是一个未知分布的期望奖励，玩家可以从中采样奖励。针对该问题，我们基于Track-and-Stop策略开发了一种具有渐近最优样本复杂度的$δ$-正确序贯采样算法。我们证明，对D-Tracking臂选择策略进行基于比率的修改，可显著改善实际样本复杂度，同时将每轮计算成本从臂数量的线性降低至对数级。