Diffusion Language Models (DLMs) enable fast generation, yet training large DLMs from scratch is costly. As a practical shortcut, adapting off-the-shelf Auto-Regressive (AR) model weights into a DLM could quickly equip the DLM with strong long-context generation capabilies. Prior "adaptation" attempts either modify logits or randomly grow attention masks to Full-Sequence diffusion, or simply transplant AR weights into a Block-Diffusion recipe, leaving two key questions unaddressed: where is the final destination of adaptation, and how to adapt better? For manifold benefits, we reframe the whole AR-to-DLM adaptation under the Block-Diffusion paradigm, transitioning from block size 1 to the final Block-Diffusion state. Concretely, the principled pathway of adaptation is designed as follows: we keep a context-causal path where causal attention is kept in the prefix, an efficient parallel adaptation procedure where an AR guidance is maintained, and gradual increment of the generation block size for a smoother transition. Built on these components, the adaptation is proved competitive on various models at different scales. With better adaptation, we propose NBDiff-7B that could inherit the long-context modeling and reasoning capabilities, and achieve state-of-the-art performance among the 7B-class DLMs. Codes: https://github.com/YuchuanTian/NBDiff.

翻译：扩散语言模型（DLMs）能够实现快速生成，但从头训练大型DLMs成本高昂。作为一种实用捷径，将现成的自回归（AR）模型权重适配到DLM中，可以快速赋予DLM强大的长上下文生成能力。先前的“适应”尝试要么修改逻辑值或将注意力掩码随机扩展至全序列扩散，要么简单地将AR权重移植到区块扩散方案中，留下了两个关键问题未解决：适应的最终目标是什么？以及如何更好地进行适应？为了获得多方面优势，我们在区块扩散范式下重新构建了整个AR到DLM的适应过程，从区块大小为1过渡到最终的区块扩散状态。具体而言，原则性的适应路径设计如下：我们保留一个上下文因果路径，其中前缀部分保持因果注意力；采用高效的并行适应程序，维持AR引导；并逐步增加生成区块大小以实现更平滑的过渡。基于这些组件，该适应方法在不同规模的多种模型上被证明具有竞争力。通过更好的适应，我们提出了NBDiff-7B模型，它能够继承长上下文建模和推理能力，并在7B级DLMs中实现最先进的性能。代码：https://github.com/YuchuanTian/NBDiff。