Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes the generative (reverse) process with factorized distributions, which makes it difficult for the model to learn the target process in a small number of steps and necessitates a long, computationally expensive sampling procedure. To reduce the gap between the target and model distributions and enable few-step generation, we propose Forward-Learned Discrete Diffusion (FLDD), which introduces discrete diffusion with a learnable forward (noising) process. Rather than fixing a Markovian forward chain, we adopt a non-Markovian formulation with learnable marginal and posterior distributions. This allows the generative process to remain factorized while matching the target defined by the noising process. We train all parameters end-to-end under the standard variational objective. Experiments on various benchmarks show that, for a given number of sampling steps, our approach produces a higher quality samples than conventional discrete diffusion models using the same reverse parameterization.

翻译：离散扩散模型是一类强大的生成模型，在众多领域展现出优异性能。然而，出于效率考量，离散扩散通常采用因子化分布参数化生成（逆向）过程，这使得模型难以在少量步骤内学习目标过程，从而需要冗长且计算昂贵的采样流程。为缩小目标分布与模型分布之间的差距并实现少步生成，我们提出前向学习离散扩散（FLDD），该模型引入具有可学习前向（加噪）过程的离散扩散方法。我们摒弃固定的马尔可夫前向链，采用具有可学习边际分布和后验分布的非马尔可夫公式。这使得生成过程在保持因子化的同时，能够匹配由加噪过程定义的目标分布。所有参数均在标准变分目标下进行端到端训练。在多个基准测试上的实验表明，在给定采样步数的情况下，我们的方法相比采用相同逆向参数化的传统离散扩散模型，能生成更高质量的样本。