We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions. In contrast to Gibbs sampling-based methods, DLP is able to update all coordinates in parallel in a single step and the magnitude of changes is controlled by a stepsize. This allows a cheap and efficient exploration in the space of high-dimensional and strongly correlated variables. We prove the efficiency of DLP by showing that the asymptotic bias of its stationary distribution is zero for log-quadratic distributions, and is small for distributions that are close to being log-quadratic. With DLP, we develop several variants of sampling algorithms, including unadjusted, Metropolis-adjusted, stochastic and preconditioned versions. DLP outperforms many popular alternatives on a wide variety of tasks, including Ising models, restricted Boltzmann machines, deep energy-based models, binary neural networks and language generation.

翻译：我们提出了离散的Langevin建议(DLP),这是一个简单且可缩放的基于梯度的建议,用于取样复杂的高维离散分布。与Gibbs基于取样的方法不同,DLP能够平行地同步更新所有坐标,而变化的规模则通过一个阶梯化来控制。这样就可以在高维和密切相关的变量空间中进行廉价而高效的探索。我们通过显示其定点分布的无症状偏差对正对流分布为零,对于接近于正对流的分布而言,DLP是很小的。我们与DLP一起开发了几种取样算法的变种,包括未经调整的、大都会调整的、随机的和先决条件的版本。DLP在包括Ising模型、受限制的Boltzmann机器、深能量基模型、双神经网络和语言生成等多种任务上,都比许多受欢迎的替代方法要低得多。