Because state-of-the-art language models are expensive to train, most practitioners must make use of one of the few publicly available language models or language model APIs. This consolidation of trust increases the potency of backdoor attacks, where an adversary tampers with a machine learning model in order to make it perform some malicious behavior on inputs that contain a predefined backdoor trigger. We show that the in-context learning ability of large language models significantly complicates the question of developing backdoor attacks, as a successful backdoor must work against various prompting strategies and should not affect the model's general purpose capabilities. We design a new attack for eliciting targeted misclassification when language models are prompted to perform a particular target task and demonstrate the feasibility of this attack by backdooring multiple large language models ranging in size from 1.3 billion to 6 billion parameters. Finally we study defenses to mitigate the potential harms of our attack: for example, while in the white-box setting we show that fine-tuning models for as few as 500 steps suffices to remove the backdoor behavior, in the black-box setting we are unable to develop a successful defense that relies on prompt engineering alone.

翻译：由于最先进的语言模型训练成本高昂，大多数从业者必须使用少数公开可用的语言模型或语言模型API。这种信任的集中化增强了后门攻击的效力——攻击者篡改机器学习模型，使其对包含预定义后门触发器的输入执行恶意行为。我们证明，大语言模型的上下文学习能力显著复杂化了后门攻击的开发问题，因为成功的后门必须能对抗多种提示策略，且不应影响模型的通用能力。我们设计了一种新型攻击方法，当语言模型被提示执行特定目标任务时，能引发目标性错误分类，并通过在参数量从13亿到60亿不等的多个大语言模型上植入后门，验证了该攻击的可行性。最后，我们研究了缓解攻击潜在危害的防御措施：例如在白盒环境下，仅需对模型进行500步微调即可消除后门行为；但在黑盒环境下，我们未能开发出仅依赖提示工程的有效防御方法。