Diffusion large language models (dLLMs) offer parallel decoding and bidirectional context, but state-of-the-art dLLMs require billions of parameters for competitive performance. While existing distillation methods for dLLMs reduce inference steps within a single architecture, none address cross-architecture knowledge transfer, in which the teacher and student differ in architecture, attention mechanism, and tokenizer. We present TIDE, the first framework for cross-architecture dLLM distillation, comprising three modular components: (1) TIDAL, which jointly modulates distillation strength across training progress and diffusion timestep to account for the teacher's noise-dependent reliability; (2) CompDemo, which enriches the teacher's context via complementary mask splitting to improve predictions under heavy masking; and (3) Reverse CALM, a cross-tokenizer objective that inverts chunk-level likelihood matching, yielding bounded gradients and dual-end noise filtering. Distilling 8B dense and 16B MoE teachers into a 0.6B student via two heterogeneous pipelines outperforms the baseline by an average of 1.53 points across eight benchmarks, yielding notable gains in code generation, where HumanEval scores reach 48.78 compared to 32.3 for the AR baseline.

翻译：扩散大语言模型（dLLMs）能够实现并行解码和双向上下文，但最先进的dLLMs需要数十亿参数才能达到竞争性性能。现有针对dLLMs的蒸馏方法仅能在单一架构内减少推理步骤，尚未涉及跨架构知识迁移（即教师模型与学生模型在架构、注意力机制和分词器上均存在差异）。我们提出TIDE——首个面向跨架构dLLM蒸馏的框架，包含三个模块化组件：（1）TIDAL，联合调控训练进程与扩散时间步上的蒸馏强度，以应对教师模型依赖噪声的可靠性变化；（2）CompDemo，通过互补掩码分割丰富教师模型的上下文信息，从而改进高掩码率下的预测质量；（3）反向CALM，一种跨分词器目标函数，通过反转块级似然匹配产生有界梯度和双端噪声过滤。通过两条异质流水线将80亿参数密集型和160亿参数MoE教师模型蒸馏至6亿参数学生模型，在八项基准测试中平均超越基线1.53分，尤其在代码生成任务中取得显著提升——HumanEval得分达到48.78，而自回归基线为32.3。