Generative models for learning combinatorial structures have transformative impacts in many applications. However, existing approaches fail to offer efficient and accurate learning results. Because of the highly intractable nature of the gradient estimation of the learning objective subject to combinatorial constraints. Existing gradient estimation methods would easily run into exponential time/memory space, or incur huge estimation errors due to improper approximation. We develop NEural Lovasz Sampler (Nelson), a neural network based on Lov\'asz Local Lemma (LLL). We show it guarantees to generate samples satisfying combinatorial constraints from the distribution of the constrained Markov Random Fields model (MRF) under certain conditions. We further present a fully differentiable contrastive-divergence-based learning framework on constrained MRF (Nelson-CD). Meanwhile, Nelson-CD being fully differentiable allows us to take advantage of the parallel computing power of GPUs, resulting in great efficiency. Experimental results on three real-world combinatorial problems reveal that Nelson learns to generate 100% valid structures. In comparison, baselines either time out on large-size data sets or fail to generate valid structures, whereas Nelson scales much better with problem size. In addition, Nelson outperforms baselines in various learning metrics, such as log-likelihood and MAP scores.

翻译：生成模型用于学习组合结构在许多应用中具有变革性影响，但现有方法无法提供高效且准确的学习结果，这是因为在组合约束下学习目标的梯度估计具有高度难解性。现有梯度估计方法容易陷入指数级时间/内存空间，或因不当近似产生巨大估计误差。我们提出了基于洛夫·局部引理的神经网络——NEural Lovasz Sampler（Nelson），证明了在特定条件下，它能保证从受约束马尔可夫随机场模型的分布中生成满足组合约束的样本。进一步地，我们提出了一个完全可微的基于对比散度的约束马尔可夫随机场学习框架（Nelson-CD）。同时，Nelson-CD的完全可微性使我们能利用GPU的并行计算能力，从而显著提升效率。在三个真实世界组合问题上的实验结果表明，Nelson能够生成100%的有效结构。相比之下，基线方法要么在大型数据集上超时，要么无法生成有效结构，而Nelson在问题规模扩展时表现出更好的可扩展性。此外，Nelson在对数似然和最大后验分数等多项学习指标上均优于基线方法。