Masked diffusion language models (MDLMs) generate text via iterative masked-token denoising, enabling mask-parallel decoding and distinct controllability and efficiency tradeoffs from autoregressive LLMs. Yet, efficient representation-level mechanisms for inference-time control in MDLMs remain largely unexplored. To address this gap, we introduce an activation steering primitive for MDLMs: we extract a single low-dimensional direction from contrastive prompt sets using one prompt-only forward pass, and apply a global intervention on residual-stream activations throughout reverse diffusion, without performing optimization or altering the diffusion sampling procedure. Using safety refusal as a deployment-relevant case study, we find that refusal behavior in multiple MDLMs is governed by a consistent, approximately one-dimensional activation subspace. Applying the corresponding direction yields large and systematic behavioral shifts and is substantially more effective than prompt-based and optimization-based baselines. We further uncover diffusion-specific accessibility: effective directions can be extracted not only from post-instruction tokens, but also from pre-instruction tokens that are typically ineffective in autoregressive models due to causal attention. Ablations localize maximal leverage to early denoising steps and mid-to-late transformer layers, with early diffusion blocks contributing disproportionately. Finally, in an MDLM trained on English and Chinese, extracted directions transfer strongly between English and Chinese, but do not reliably generalize to an autoregressive architecture, highlighting architecture-dependent representations of safety constraints.
翻译:遮蔽扩散语言模型(MDLMs)通过迭代式遮蔽词元去噪生成文本,支持掩码并行解码,并在可控性和效率权衡方面与自回归大语言模型形成差异。然而,针对MDLMs推理时控制的高效表征层级机制仍鲜有探索。为弥合这一研究空白,我们提出面向MDLMs的激活引导原型:通过单次仅需提示的前向传播,从对比提示集合中提取单一低维方向,并在逆向扩散过程中对残差流激活施加全局干预,无需执行优化或改变扩散采样流程。以安全拒绝作为部署相关案例研究,我们发现多个MDLMs中的拒绝行为受控于一个一致、近似一维的激活子空间。应用对应方向可引发显著且系统性的行为偏移,其效果远超基于提示和优化的基线方法。我们进一步揭示扩散模型特有的可访问性:有效方向不仅可从指令后词元提取,还可从指令前词元提取——后者因因果注意力机制在自回归模型中通常无效。消融实验表明,逆向扩散早期步骤与变换器中后期层具有最大调控效力,且早期扩散模块贡献比例失调。最后,在基于中英双语训练的MDLM中,提取的方向可在中英语言间实现强迁移,但无法可靠泛化至自回归架构,凸显安全约束表征的架构依赖性。