This work focuses on defending against the data poisoning based backdoor attacks, which bring in serious security threats to deep neural networks (DNNs). Specifically, given a untrustworthy training dataset, we aim to filter out potential poisoned samples, \ie, poisoned sample detection (PSD). The key solution for this task is to find a discriminative metric between clean and poisoned samples, even though there is no information about the potential poisoned samples (\eg, the attack method, the poisoning ratio). In this work, we develop an innovative detection approach from the perspective of the gradient \wrt activation (\ie, activation gradient direction, AGD) of each sample in the backdoored model trained on the untrustworthy dataset. We present an interesting observation that the circular distribution of AGDs among all samples of the target class is much more dispersed than that of one clean class. Motivated by this observation, we firstly design a novel metric called Cosine similarity Variation towards Basis Transition (CVBT) to measure the circular distribution's dispersion of each class. Then, we design a simple yet effective algorithm with identifying the target class(es) using outlier detection on CVBT scores of all classes, followed by progressively filtering of poisoned samples according to the cosine similarities of AGDs between every potential sample and a few additional clean samples. Extensive experiments under various settings verify that given very few clean samples of each class, the proposed method could filter out most poisoned samples, while avoiding filtering out clean samples, verifying its effectiveness on the PSD task. Codes are available at https://github.com/SCLBD/bdzoo2/blob/dev/detection_pretrain/agpd.py.

翻译：本文聚焦于防御基于数据投毒的后门攻击，此类攻击对深度神经网络（DNN）构成严重安全威胁。具体而言，针对不可信训练数据集，我们旨在滤除潜在毒化样本，即毒化样本检测（PSD）。该任务的核心解决方案在于寻找干净样本与毒化样本之间的可区分度量，即便缺乏关于潜在毒化样本（如攻击方法、投毒比例）的任何先验信息。本文从不可信数据集训练的后门模型中每个样本的激活梯度方向（AGD）视角出发，提出了一种创新性检测方法。我们观察到目标类所有样本的AGD圆环分布比干净类的圆环分布更为分散。基于此观察，我们首先设计了一种名为"基变换余弦相似度变异（CVBT）"的创新度量指标，用以衡量各类别圆环分布的离散程度。进而设计了一种简洁高效的算法：通过所有类别CVBT得分的异常检测识别目标类别，再根据潜在样本与少量干净样本间AGD余弦相似度逐步过滤毒化样本。多场景下的实验验证表明，在每类仅需极少量干净样本的条件下，该方法能在避免误删干净样本的同时滤除大部分毒化样本，充分验证了其在PSD任务中的有效性。代码已公开于https://github.com/SCLBD/bdzoo2/blob/dev/detection_pretrain/agpd.py。