Token-to-token (T2T) editing lets LLaDA2.1 revise committed tokens during block-diffusion decoding. The released recipe trains this editor on random vocabulary corruptions, but at inference the editor sees the model's own fluent, high-confidence draft errors instead. We study this training-inference mismatch and propose self-generated T2T, which performs a no-gradient draft pass, fills masked positions with predicted tokens, and supervises recovery in a second pass under these self-generated corruptions. We implement the update as a short LoRA continued-pretraining pass on LLaDA2.1-mini and evaluate on several benchmarks under the official Q-Mode T2T procedure with unchanged inference parameters. The method generally improves accuracy while reducing T2T edit intensity, mitigating failure modes such as final-digit transcription errors after otherwise correct reasoning and excessive self-correction before short factual answers.
翻译:Token到Token(T2T)编辑允许LLaDA2.1在块扩散解码过程中修正已承诺的Token。已发布的方案通常训练该编辑器应对随机词汇损坏,但在推理时,编辑器面对的是模型自身生成的流畅且高置信度的草稿错误。我们研究了这种训练与推理之间的不匹配,并提出了自生成T2T方法,该方法执行一次不涉及梯度的草稿传递,用预测的Token填充被掩码的位置,并在第二次传递中在自生成错误条件下监督恢复过程。我们通过LLaDA2.1-mini上的简短LoRA连续预训练实现更新,并在官方Q-Mode T2T流程中使用不变的推理参数,在多个基准测试上进行评估。该方法在降低T2T编辑强度的同时普遍提升准确率,同时缓解了若干典型失败模式,例如正确推理后出现最终数字转录错误,以及短事实答案前的过度自我修正。