Self-correction is an effective technique for maintaining parallel sampling in discrete diffusion models with minimal performance degradation. Prior work has explored self-correction at inference time or during post-training; however, such approaches often suffer from limited generalization and may impair reasoning performance. GIDD pioneers pretraining-based self-correction via a multi-step BERT-style uniform-absorbing objective. However, GIDD relies on a continuous interpolation-based pipeline with opaque interactions between uniform transitions and absorbing masks, which complicates hyperparameter tuning and hinders practical performance. In this work, we propose a Self-Correcting Discrete Diffusion (SCDD) model to reformulate pretrained self-correction with explicit state transitions and learn directly in discrete time. Our framework also simplifies the training noise schedule, eliminates a redundant remasking step, and relies exclusively on uniform transitions to learn self-correction. Experiments at the GPT-2 scale demonstrate that our method enables more efficient parallel decoding while preserving generation quality.

翻译：自纠正是保持离散扩散模型并行采样性能的有效技术，且几乎不造成生成质量下降。先前研究已在推理阶段或后训练过程中探索自纠正方法，然而这类方法通常泛化能力有限，且可能损害推理性能。GIDD通过多步BERT风格的均匀-吸收目标率先提出基于预训练的自纠正方案，但其依赖基于连续插值的流水线，其中均匀转移与吸收掩码的交互机制不透明，导致超参数调优复杂并影响实际性能。本文提出自纠正离散扩散(SCDD)模型，以显式状态转移重新定义预训练式自纠正，并直接在离散时间域进行学习。本框架同时简化训练噪声调度机制，消除冗余的重掩码步骤，仅依赖均匀转移实现自纠正能力学习。GPT-2规模实验表明，本方法在保证生成质量的同时，实现了更高效的并行解码。