The well-known Gumbel-Max Trick for sampling elements from a categorical distribution (or more generally a non-negative vector) and its variants have been widely used in areas such as machine learning and information retrieval. To sample a random element $i$ in proportion to its positive weight $v_i$, the Gumbel-Max Trick first computes a Gumbel random variable $g_i$ for each positive weight element $i$, and then samples the element $i$ with the largest value of $g_i+\ln v_i$. Recently, applications including similarity estimation and weighted cardinality estimation require to generate $k$ independent Gumbel-Max variables from high dimensional vectors. However, it is computationally expensive for a large $k$ (e.g., hundreds or even thousands) when using the traditional Gumbel-Max Trick. To solve this problem, we propose a novel algorithm, FastGM, which reduces the time complexity from $O(kn^+)$ to $O(k \ln k + n^+)$, where $n^+$ is the number of positive elements in the vector of interest. FastGM stops the procedure of Gumbel random variables computing for many elements, especially for those with small weights. We perform experiments on a variety of real-world datasets and the experimental results demonstrate that FastGM is orders of magnitude faster than state-of-the-art methods without sacrificing accuracy or incurring additional expenses.

翻译：众所周知的用于从类别分布（或更一般地，从非负向量）中采样元素的Gumbel-Max技巧及其变体已被广泛应用于机器学习与信息检索等领域。为了按正权重$v_i$的比例采样随机元素$i$，Gumbel-Max技巧首先为每个正权重元素$i$计算一个Gumbel随机变量$g_i$，然后采样具有最大$g_i+\ln v_i$值的元素$i$。近年来，包括相似性估计与加权基数估计在内的应用需要从高维向量中生成$k$个独立的Gumbel-Max变量。然而，当使用传统Gumbel-Max技巧时，对于较大的$k$（例如数百甚至数千），其计算代价高昂。为解决该问题，我们提出一种新颖算法FastGM，将时间复杂度从$O(kn^+)$降低至$O(k \ln k + n^+)$，其中$n^+$为所关注向量中正元素的数量。FastGM对大量元素（尤其是权重较小的元素）停止了Gumbel随机变量的计算过程。我们在多种真实世界数据集上进行了实验，实验结果表明，FastGM在不牺牲精度或引入额外开销的情况下，比现有最优方法快数个数量级。