Variational flows allow practitioners to learn complex continuous distributions, but approximating discrete distributions remains a challenge. Current methodologies typically embed the discrete target in a continuous space - usually via continuous relaxation or dequantization - and then apply a continuous flow. These approaches involve a surrogate target that may not capture the original discrete target, might have biased or unstable gradients, and can create a difficult optimization problem. In this work, we develop a variational flow family for discrete distributions without any continuous embedding. First, we develop a measure-preserving and discrete (MAD) invertible map that leaves the discrete target invariant, and then create a mixed variational flow (MAD Mix) based on that map. Our family provides access to i.i.d. sampling and density evaluation with virtually no tuning effort. We also develop an extension to MAD Mix that handles joint discrete and continuous models. Our experiments suggest that MAD Mix produces more reliable approximations than continuous-embedding flows while being significantly faster to train.

翻译：变分流使从业者能够学习复杂的连续分布，但近似离散分布仍是挑战。当前方法通常将离散目标嵌入连续空间——通常通过连续松弛或去量化——然后应用连续流。这些方法涉及可能无法捕捉原始离散目标的替代目标，存在梯度有偏或不稳定的问题，并可能产生困难的优化问题。在本工作中，我们开发了一种无需任何连续嵌入的离散分布变分流族。首先，我们构造了一种保持测度且离散（MAD）的可逆映射，该映射能保持离散目标不变，并基于该映射创建了混合变分流（MAD Mix）。我们的流族几乎无需调参即可实现独立同分布采样和密度评估。我们还开发了MAD Mix的扩展版本以处理联合离散连续模型。实验表明，MAD Mix比连续嵌入流能产生更可靠的近似结果，同时训练速度显著更快。