We introduce $\texttt{PairFlow}$, a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiring a pretrained teacher. DFMs have recently emerged as a new class of generative models for discrete data, offering strong performance. However, they suffer from slow sampling due to their iterative nature. Existing acceleration methods largely depend on finetuning, which introduces substantial additional training overhead. $\texttt{PairFlow}$ addresses this issue with a lightweight preprocessing step. Inspired by ReFlow and its extension to DFMs, we train DFMs from coupled samples of source and target distributions, without requiring any pretrained teacher. At the core of our approach is a closed-form inversion for DFMs, which allows efficient construction of paired source-target samples. Despite its extremely low cost, taking only up to 1.7% of the compute needed for full model training, $\texttt{PairFlow}$ matches or even surpasses the performance of two-stage training involving finetuning. Furthermore, models trained with our framework provide stronger base models for subsequent distillation, yielding further acceleration after finetuning. Experiments on molecular data as well as binary and RGB images demonstrate the broad applicability and effectiveness of our approach.

翻译：本文提出 $\texttt{PairFlow}$，一种用于训练离散流模型（DFMs）的轻量级预处理步骤，可在无需预训练教师模型的情况下实现少步采样。离散流模型是近期出现的一类针对离散数据的生成模型，具有优异的生成性能。然而，由于其迭代采样的特性，这类模型存在采样速度缓慢的问题。现有的加速方法主要依赖于微调，这带来了显著的额外训练开销。$\texttt{PairFlow}$ 通过一种轻量级预处理步骤解决了这一问题。受 ReFlow 及其在离散流模型中扩展的启发，我们直接从源分布与目标分布的耦合样本中训练离散流模型，无需任何预训练的教师模型。我们方法的核心在于为离散流模型推导出的闭式逆变换，该变换能够高效地构建成对的源-目标样本。尽管计算成本极低，仅需完整模型训练计算量的 1.7%，$\texttt{PairFlow}$ 的性能却能够匹配甚至超越包含微调阶段的两阶段训练方法。此外，基于我们框架训练的模型能为后续的蒸馏提供更优的基础模型，进而在微调后实现进一步的加速。在分子数据、二值图像及 RGB 图像上的实验验证了我们方法的广泛适用性与有效性。