Estimating the gradients of stochastic nodes in stochastic computational graphs is one of the crucial research questions in the deep generative modeling community, which enables the gradient descent optimization on neural network parameters. Stochastic gradient estimators of discrete random variables are widely explored, for example, Gumbel-Softmax reparameterization trick for Bernoulli and categorical distributions. Meanwhile, other discrete distribution cases such as the Poisson, geometric, binomial, multinomial, negative binomial, etc. have not been explored. This paper proposes a generalized version of the Gumbel-Softmax estimator, which is able to reparameterize generic discrete distributions, not restricted to the Bernoulli and the categorical. The proposed estimator utilizes the truncation of discrete random variables, the Gumbel-Softmax trick, and a special form of linear transformation. Our experiments consist of (1) synthetic examples and applications on VAE, which show the efficacy of our methods; and (2) topic models, which demonstrate the value of the proposed estimation in practice.

翻译：估计透视计算图中的随机节点梯度是深基因模型界的关键研究问题之一,它使得神经网络参数的梯度下降优化成为了神经网络参数的梯度优化。 广泛探索了离散随机变量的悬浮梯度估计器, 例如 Bernoulli 和 绝对分布 。 同时, 其它离散分布案例, 如 Poisson 、 几何、 二流、 多数值、 负二元等, 尚未被探索 。 本文提出了 Gumbel- Softmax 估计器的通用版本, 它可以对不局限于 Bernoulli 和 直径的通用离散分布进行重新校准。 拟议的估计器使用了离散随机变量、 Gumbel- 软体魔术 和 线性转换的特殊形式 。 我们的实验包括 (1) 合成示例和应用 VAE, 显示我们方法的功效; (2) 主题模型, 展示了拟议的估计方法的价值。