Diffusion Language Models (DLMs) offer a promising alternative for language modeling by enabling parallel decoding through iterative refinement. However, most DLMs rely on hard binary masking and discrete token assignments, which hinder the revision of early decisions and underutilize intermediate probabilistic representations. In this paper, we propose EvoToken-DLM, a novel diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. EvoToken-DLM enables a progressive transition from masked states to discrete outputs, supporting revisable decoding. To effectively support this evolution, we introduce continuous trajectory supervision, which aligns training objectives with iterative probabilistic updates. Extensive experiments across multiple benchmarks show that EvoToken-DLM consistently achieves superior performance, outperforming strong diffusion-based and masked DLM baselines. Project webpage: https://aim-uofa.github.io/EvoTokenDLM.

翻译：扩散语言模型通过迭代优化实现并行解码，为语言建模提供了一种前景广阔的替代方案。然而，现有扩散语言模型大多依赖硬二元掩码和离散词元分配机制，这限制了早期决策的可修正性，且未能充分利用中间概率表示。本文提出EvoToken-DLM，一种创新的基于扩散的语言建模方法，该方法将硬二元掩码替换为演化的软词元分布。EvoToken-DLM实现了从掩码状态到离散输出的渐进式过渡，支持可修正的解码过程。为有效支撑此演化机制，我们引入了连续轨迹监督方法，使训练目标与迭代概率更新保持一致。在多个基准测试上的广泛实验表明，EvoToken-DLM始终取得卓越性能，显著优于主流的基于扩散及掩码的扩散语言模型基线。项目网页：https://aim-uofa.github.io/EvoTokenDLM。