The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model's performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field.
翻译:机器学习的成功得益于计算能力和大规模训练数据集的日益增长。训练数据用于学习新模型或更新现有模型,其基本假设是这些数据能充分代表测试阶段将会遇到的数据。然而,投毒攻击(一种通过操纵训练数据以在测试阶段损害模型性能的攻击)对这一假设构成了挑战。尽管投毒已被视为工业应用中的相关威胁,且已提出多种不同类型的攻击与防御方法,但该领域仍缺乏完整的系统化梳理与批判性综述。本综述对机器学习中的投毒攻击与防御进行了全面系统化整理,系统回顾了过去15年间该领域发表的100余篇论文。我们首先对当前威胁模型与攻击进行分类,进而据此组织现有防御措施。虽然主要关注计算机视觉应用,但我们的系统化框架同样适用于其他数据模态的最新攻击与防御方法。最后,我们讨论了投毒研究的现有资源,并揭示了该研究领域当前的局限性与开放性问题。