Deep Neural Networks (DNNs) have led to unprecedented progress in various natural language processing (NLP) tasks. Owing to limited data and computation resources, using third-party data and models has become a new paradigm for adapting various tasks. However, research shows that it has some potential security vulnerabilities because attackers can manipulate the training process and data source. Such a way can set specific triggers, making the model exhibit expected behaviors that have little inferior influence on the model's performance for primitive tasks, called backdoor attacks. Hence, it could have dire consequences, especially considering that the backdoor attack surfaces are broad. To get a precise grasp and understanding of this problem, a systematic and comprehensive review is required to confront various security challenges from different phases and attack purposes. Additionally, there is a dearth of analysis and comparison of the various emerging backdoor countermeasures in this situation. In this paper, we conduct a timely review of backdoor attacks and countermeasures to sound the red alarm for the NLP security community. According to the affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into three categorizations: attacking pre-trained model with fine-tuning (APMF) or prompt-tuning (APMP), and attacking final model with training (AFMT), where AFMT can be subdivided into different attack aims. Thus, attacks under each categorization are combed. The countermeasures are categorized into two general classes: sample inspection and model inspection. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. An attacker can intelligently bypass existing defenses with a more invisible attack. ......
翻译:深度神经网络(DNNs)已在各种自然语言处理(NLP)任务中取得前所未有的进展。由于数据和计算资源有限,使用第三方数据和模型已成为适应不同任务的新范式。然而,研究表明,攻击者可能操纵训练过程和数据源,从而导致潜在的安全漏洞。这类攻击可通过设置特定触发器,使模型表现出预期行为,而对原始任务的模型性能影响甚微,此类攻击被称为后门攻击。因此,考虑到后门攻击面广泛,其可能造成严重后果。为准确理解和把握这一问题,需要系统且全面的综述来应对不同阶段和攻击目的所产生的各种安全挑战。此外,当前对各种新兴后门防御措施的分析与比较仍显不足。本文及时回顾了后门攻击与防御措施,为NLP安全领域敲响警钟。根据机器学习流水线受影响的阶段,攻击面被识别为广泛分布,并形式化为三类:攻击微调预训练模型(APMF)、攻击提示调优预训练模型(APMP),以及攻击最终模型训练过程(AFMT),其中AFMT可进一步细分为不同攻击目标。据此,对每类攻击进行了梳理。防御措施被划分为两大类别:样本检测和模型检测。总体而言,防御方面的研究远落后于攻击方面,尚无单一防御能阻止所有类型的后门攻击。攻击者可通过更隐蔽的攻击方式智能地绕过现有防御手段。……