Deep Neural Networks (DNNs) have led to unprecedented progress in various natural language processing (NLP) tasks. Owing to limited data and computation resources, using third-party data and models has become a new paradigm for adapting various tasks. However, research shows that it has some potential security vulnerabilities because attackers can manipulate the training process and data source. Such a way can set specific triggers, making the model exhibit expected behaviors that have little inferior influence on the model's performance for primitive tasks, called backdoor attacks. Hence, it could have dire consequences, especially considering that the backdoor attack surfaces are broad. To get a precise grasp and understanding of this problem, a systematic and comprehensive review is required to confront various security challenges from different phases and attack purposes. Additionally, there is a dearth of analysis and comparison of the various emerging backdoor countermeasures in this situation. In this paper, we conduct a timely review of backdoor attacks and countermeasures to sound the red alarm for the NLP security community. According to the affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into three categorizations: attacking pre-trained model with fine-tuning (APMF) or prompt-tuning (APMP), and attacking final model with training (AFMT), where AFMT can be subdivided into different attack aims. Thus, attacks under each categorization are combed. The countermeasures are categorized into two general classes: sample inspection and model inspection. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. An attacker can intelligently bypass existing defenses with a more invisible attack. ......
翻译:深度神经网络(DNNs)在各类自然语言处理(NLP)任务中取得了前所未有的进展。由于数据和计算资源有限,使用第三方数据和模型已成为适配不同任务的新范式。然而,研究表明,这种方式存在潜在安全漏洞,因为攻击者可以操纵训练过程和数据源。通过设置特定触发器,使模型表现出对原始任务性能影响甚微的预期行为,这种攻击方式被称为后门攻击。鉴于后门攻击面广泛,可能造成严重后果。为精准把握和理解该问题,需进行系统全面的综述,以应对不同阶段和攻击目的带来的各类安全挑战。此外,当前对各类新兴后门防御措施的分析与比较仍显不足。本文对后门攻击与防御措施进行及时综述,为NLP安全社区敲响警钟。根据机器学习流水线受影响阶段,识别出宽广的攻击面,并将其形式化为三类:攻击预训练模型与微调(APMF)、攻击预训练模型与提示调优(APMP),以及攻击最终模型与训练(AFMT),其中AFMT可根据不同攻击目标进一步细分。据此,对每类攻击进行了梳理。防御措施则分为两大类别:样本检测与模型检测。总体而言,防御侧研究远落后于攻击侧,尚无单一防御机制能阻止所有类型的后门攻击。攻击者可通过更具隐蔽性的攻击智能绕过现有防御。……