Non-autoregressive (NAR) language models offer notable efficiency in text generation by circumventing the sequential bottleneck of autoregressive decoding. However, accurately modeling dependencies in discrete sequences remains challenging in this paradigm. In this work, we advance the field of NAR generation by applying conditional flow matching (CFM) methods grounded in geometrically principled interpolation, specifically leveraging Kullback-Leibler (KL) divergence geodesics, which correspond to linear interpolation in logit space. We rigorously establish that maximizing conditional likelihood in this setting precisely recovers the flow matching velocity field, supplying the theoretical justification for this approach in sequence modeling. To address practical performance gaps of basic inference, we propose a novel empirical sampling strategy that iteratively denoises and re-noises, along with a hybrid scheme that integrates our sampling method with basic procedure. Across unconditional and conditional text and code infilling, the approach improves perplexity and downstream metrics over prior NAR baselines under matched settings.

翻译：非自回归（NAR）语言模型通过规避自回归解码中的顺序瓶颈，在文本生成中实现了显著的效率提升。然而，在此范式下准确建模离散序列中的依赖关系仍具有挑战性。本工作通过应用基于几何原理插值的条件流匹配（CFM）方法，特别是利用对应于logit空间中线性插值的KL散度测地线，推动了NAR生成领域的发展。我们严格论证了在此设定下最大化条件似然能够精确恢复流匹配速度场，为该方法在序列建模中的应用提供了理论依据。为解决基础推理在实际性能上的差距，我们提出了一种新的经验采样策略——通过迭代去噪与再噪化过程，并设计了一种将该采样方法与基础流程相结合的混合方案。在无条件和有条件的文本及代码填充任务中，该方法在匹配设置下相较于先前的NAR基线，在困惑度和下游指标上均取得了改进。