To ensure AI safety, instruction-tuned Large Language Models (LLMs) are specifically trained to ensure alignment, which refers to making models behave in accordance with human intentions. While these models have demonstrated commendable results on various safety benchmarks, the vulnerability of their safety alignment has not been extensively studied. This is particularly troubling given the potential harm that LLMs can inflict. Existing attack methods on LLMs often rely on poisoned training data or the injection of malicious prompts. These approaches compromise the stealthiness and generalizability of the attacks, making them susceptible to detection. Additionally, these models often demand substantial computational resources for implementation, making them less practical for real-world applications. In this work, we introduce a novel attack framework, called Backdoor Activation Attack, which injects trojan steering vectors into the activation layers of LLMs. These malicious steering vectors can be triggered at inference time to steer the models toward attacker-desired behaviors by manipulating their activations. In particular, the steering vectors are generated by taking the difference between benign and malicious activations. Then, the most effective steering vector is selected and added to the forward passes of the LLMs. Our experiment results on four primary alignment tasks show that our proposed method is highly effective and adds little or no overhead to attack efficiency. Additionally, we discuss potential countermeasures against such activation attacks. Our code and data are available at https://email-haoran-for-link. Warning: this paper contains content that can be offensive or upsetting.

翻译：为确保人工智能安全，经过指令微调的大型语言模型（LLMs）通过专门训练实现安全对齐，即确保模型行为符合人类意图。尽管这些模型在各类安全基准测试中表现优异，但其安全对齐机制的脆弱性尚未得到充分研究。考虑到大型语言模型可能造成的潜在危害，这一问题尤为令人担忧。现有针对LLMs的攻击方法通常依赖于投毒训练数据或注入恶意提示，这些方法削弱了攻击的隐蔽性与泛化能力，使其容易被检测。此外，这些方法通常需要大量计算资源才能实施，降低了在实际应用中的可行性。本研究提出一种名为"后门激活攻击"的新型攻击框架，通过向LLMs的激活层注入特洛伊木马式操控向量。这些恶意操控向量可在推理阶段被触发，通过操纵模型激活值引导其产生攻击者期望的行为。具体而言，操控向量通过计算良性激活与恶意激活之间的差异生成，随后选取最具效力的操控向量注入到LLMs的前向传播过程中。我们在四项主要对齐任务上的实验结果表明，该方法具有极高攻击效力，且几乎不增加攻击效率开销。此外，本文还讨论了针对此类激活攻击的潜在防御策略。我们的代码与数据已公开于https://email-haoran-for-link。警告：本文包含可能引起不适的冒犯性内容。