Deep Neural Networks (DNNs) have led to unprecedented progress in various natural language processing (NLP) tasks. Owing to limited data and computation resources, using third-party data and models has become a new paradigm for adapting various tasks. However, research shows that it has some potential security vulnerabilities because attackers can manipulate the training process and data source. Such a way can set specific triggers, making the model exhibit expected behaviors that have little inferior influence on the model's performance for primitive tasks, called backdoor attacks. Hence, it could have dire consequences, especially considering that the backdoor attack surfaces are broad. To get a precise grasp and understanding of this problem, a systematic and comprehensive review is required to confront various security challenges from different phases and attack purposes. Additionally, there is a dearth of analysis and comparison of the various emerging backdoor countermeasures in this situation.In this paper, we conduct a timely review of backdoor attacks and countermeasures to sound the red alarm for the NLP security community. According to the affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into three categorizations: attacking pre-trained model with fine-tuning (APMF) or prompt-tuning (APMP), and attacking final model with training (AFMT), where AFMT can be subdivided into different attack aims. Thus, attacks under each categorization are combed. The countermeasures are categorized into two general classes: sample inspection and model inspection. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. An attacker can intelligently bypass existing defenses with a more invisible attack. ......
翻译:深度神经网络(DNNs)在各类自然语言处理(NLP)任务中取得了前所未有的进展。由于数据和计算资源的限制,使用第三方数据和模型已成为适配不同任务的新范式。然而,研究表明这种做法存在潜在的安全漏洞,因为攻击者能够操控训练过程和数据源。通过设置特定触发器,攻击者可使模型表现出预期行为,且该行为对模型在原任务上的性能影响极小,此类攻击称为后门攻击。考虑到后门攻击面广泛,其可能造成严重后果。为精准把握和理解该问题,需要系统全面的综述来应对不同阶段和攻击目标带来的各类安全挑战。此外,当前对各类新兴后门防御对策的分析与比较仍显不足。本文对后门攻击与防御对策进行及时综述,为NLP安全社区敲响警钟。根据机器学习流水线受影响的阶段,攻击面被识别为广泛的,并形式化为三类:攻击微调预训练模型(APMF)、攻击提示调优预训练模型(APMP),以及攻击最终模型的训练过程(AFMT),其中AFMT可进一步细分为不同攻击目标。据此,对每类攻击进行了梳理。防御对策被分为两大类:样本检测与模型检测。总体而言,防御侧的研究远落后于攻击侧,尚无单一防御能阻止所有类型的后门攻击。攻击者可通过更隐蔽的攻击手段智能绕过现有防御。