Jailbreak attacks remain a persistent threat to large language model safety. We propose Context-Conditioned Delta Steering (CC-Delta), an SAE-based defense that identifies jailbreak-relevant sparse features by comparing token-level representations of the same harmful request with and without jailbreak context. Using paired harmful/jailbreak prompts, CC-Delta selects features via statistical testing and applies inference-time mean-shift steering in SAE latent space. Across four aligned instruction-tuned models and twelve jailbreak attacks, CC-Delta achieves comparable or better safety-utility tradeoffs than baseline defenses operating in dense latent space. In particular, our method clearly outperforms dense mean-shift steering on all four models, and particularly against out-of-distribution attacks, showing that steering in sparse SAE feature space offers advantages over steering in dense activation space for jailbreak mitigation. Our results suggest off-the-shelf SAEs trained for interpretability can be repurposed as practical jailbreak defenses without task-specific training.

翻译：越狱攻击仍然是大型语言模型安全面临的持续威胁。我们提出上下文条件化Delta导向（CC-Delta），这是一种基于稀疏自编码器的防御方法，通过比较相同有害请求在包含与不包含越狱上下文时的令牌级表征，来识别与越狱相关的稀疏特征。利用配对的有害/越狱提示，CC-Delta通过统计检验选择特征，并在稀疏自编码器潜空间中进行推理时均值偏移导向。在四个经过对齐的指令调优模型和十二种越狱攻击的测试中，CC-Delta实现了与在稠密潜空间中操作的基线防御方法相当或更优的安全性-效用权衡。特别是，我们的方法在所有四个模型上均明显优于稠密均值偏移导向，尤其是在应对分布外攻击时表现突出，这表明针对越狱缓解任务，在稀疏自编码器特征空间中进行导向相比在稠密集激活空间中进行导向具有优势。我们的结果表明，为可解释性训练的开箱即用稀疏自编码器，无需针对特定任务进行训练，即可重新用作实用的越狱防御工具。