Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition and directional ablation) with two interventions derived from Iterative Nullspace Projection (INLP) -- nullspace projection and counterfactual flipping -- on five open-weight chat models, asking whether INLP can match DiM at steering refusal and whether its richer parameterisation yields more tweakable interventions. INLP counterfactual flipping is competitive with DiM directional ablation on refusal suppression, while nullspace projection is consistently weaker. Restricting INLP to the leading directions of the extracted subspace preserves most of the suppression effect at near-baseline perplexity, giving a tunable capability. Geometrically, the two INLP interventions land in qualitatively different regions of activation space: nullspace projection collapses transformed activations \emph{between} the harmful and harmless clusters, while counterfactual flipping moves them into the opposite cluster, suggesting that the model encodes the absence of a concept differently from its opposite -- an intriguing distinction that warrants further investigation in future work.

翻译：Arditi等人(2024)研究表明，安全微调对话模型中的拒绝机制由残差流中单个线性方向介导，该方向可通过有害与无害激活的差分均值法(DiM)恢复。我们在五个开源对话模型上比较了基于DiM的干预方法（激活增强与方向消融）与基于迭代零空间投影(INLP)的两种干预方法——零空间投影与反事实翻转，探究INLP能否在引导拒绝行为上媲美DiM，以及其更丰富的参数化是否能产生更具可调性的干预。实验表明：INLP反事实翻转在拒绝抑制效果上与DiM方向消融相当，而零空间投影效果持续偏弱。将INLP限制在提取子空间的主导方向，可在接近基线困惑度的条件下保留大部分抑制效果，实现可调节能力。从几何角度看，两种INLP干预方法作用于激活空间中性质不同的区域：零空间投影将变换后的激活值坍缩至有害与无害聚类之间，而反事实翻转则将其移至对立聚类，这表明模型对概念缺失与其对立面的编码存在本质差异——这一有趣区别值得未来工作深入研究。