Variational flows allow practitioners to learn complex continuous distributions, but approximating discrete distributions remains a challenge. Current methodologies typically embed the discrete target in a continuous space - usually via continuous relaxation or dequantization - and then apply a continuous flow. These approaches involve a surrogate target that may not capture the original discrete target, might have biased or unstable gradients, and can create a difficult optimization problem. In this work, we develop a variational flow family for discrete distributions without any continuous embedding. First, we develop a measure-preserving and discrete (MAD) invertible map that leaves the discrete target invariant, and then create a mixed variational flow (MAD Mix) based on that map. Our family provides access to i.i.d. sampling and density evaluation with virtually no tuning effort. We also develop an extension to MAD Mix that handles joint discrete and continuous models. Our experiments suggest that MAD Mix produces more reliable approximations than continuous-embedding flows while being significantly faster to train.

翻译：变分流使得从业者能够学习复杂的连续分布，但近似离散分布仍然是一个挑战。当前方法通常将离散目标嵌入连续空间——通常通过连续松弛或去量化——然后应用连续流。这些方法涉及一个可能无法捕捉原始离散目标的代理目标，可能产生有偏或不稳定的梯度，并可能形成难以优化的优化问题。在本工作中，我们开发了一种无需任何连续嵌入的离散分布变分流族。首先，我们构建了一个保持测度且离散（MAD）的可逆映射，该映射保持离散目标不变，然后基于该映射创建了混合变分流（MAD Mix）。我们的方法族几乎无需调参即可访问独立同分布采样和密度评估。我们还开发了MAD Mix的扩展版本，用于处理混合离散连续模型。实验表明，MAD Mix比连续嵌入流能产生更可靠的近似，同时训练速度显著更快。