Diffusion large language models (dLLMs) offer a promising alternative to autoregressive models, but their practical utility is severely hampered by slow, iterative sampling. We present SchED, a training-free, model-agnostic early-exit algorithm that aggregates full-span logit margins and halts decoding once a smooth, progress-dependent confidence threshold is met. We evaluated SchED on two dLLM families (Dream and LLaDA), in base and instruction-tuned variants across ten benchmarks spanning downstream tasks including multiple-choice question answering (MCQ), math, long-form QA/summarization, and translation. SchED delivers large, stable accelerations: on instruction-tuned models, it achieves $3.8$-$4.0\times$ speedups while retaining $99.8$-$100\%$ of the baseline score on average. On base models, SchED yields consistent speedup gains with $99.1$-$100\%$ performance retention, with up to $2.34\times$ under more aggressive settings. Using a conservative speed metric that heavily penalizes quality loss (QPS, $γ{=}4$), we show that SchED is robust and clearly outperforms prior confidence-based early-exit methods, which break down on long-form generation. An entropy analysis of the model's token predictions reveals that instruction tuning speeds up the decay of predictive entropy. By turning genuine confidence stabilization into computational savings, SchED makes dLLM decoding substantially more efficient.
翻译:扩散大语言模型(dLLMs)为自回归模型提供了一种有前景的替代方案,但其实际应用受到缓慢迭代采样的严重制约。本文提出SchED,一种无需训练、模型无关的提前退出算法,该算法通过聚合全跨度对数边际值,并在达到平滑的进度依赖置信度阈值时终止解码。我们在两个dLLM系列(Dream和LLaDA)上评估了SchED,包括基础版本和指令调优版本,覆盖了十个基准测试,涵盖多项下游任务:多项选择题回答(MCQ)、数学推理、长文本问答/摘要生成以及翻译。SchED实现了显著且稳定的加速效果:在指令调优模型上,平均获得$3.8$-$4.0$倍的加速,同时保持$99.8$-$100\\%$的基线得分;在基础模型上,SchED以$99.1$-$100\\%$的性能保留率提供一致的加速增益,在更激进的设置下最高可达$2.34$倍加速。采用严格惩罚质量损失的速度度量指标(QPS,$γ{=}4$),我们证明SchED具有鲁棒性,且明显优于先前的基于置信度的提前退出方法(后者在长文本生成任务中失效)。对模型令牌预测的熵分析表明,指令调优加速了预测熵的衰减过程。通过将真实的置信度稳定转化为计算效率提升,SchED显著提高了dLLM解码的效率。