Vision transformers (ViTs) have been successfully deployed in a variety of computer vision tasks, but they are still vulnerable to adversarial samples. Transfer-based attacks use a local model to generate adversarial samples and directly transfer them to attack a target black-box model. The high efficiency of transfer-based attacks makes it a severe security threat to ViT-based applications. Therefore, it is vital to design effective transfer-based attacks to identify the deficiencies of ViTs beforehand in security-sensitive scenarios. Existing efforts generally focus on regularizing the input gradients to stabilize the updated direction of adversarial samples. However, the variance of the back-propagated gradients in intermediate blocks of ViTs may still be large, which may make the generated adversarial samples focus on some model-specific features and get stuck in poor local optima. To overcome the shortcomings of existing approaches, we propose the Token Gradient Regularization (TGR) method. According to the structural characteristics of ViTs, TGR reduces the variance of the back-propagated gradient in each internal block of ViTs in a token-wise manner and utilizes the regularized gradient to generate adversarial samples. Extensive experiments on attacking both ViTs and CNNs confirm the superiority of our approach. Notably, compared to the state-of-the-art transfer-based attacks, our TGR offers a performance improvement of 8.8% on average.

翻译：视觉Transformer（ViTs）已成功部署于多种计算机视觉任务，但仍易受对抗样本攻击。基于迁移的攻击利用本地模型生成对抗样本，并直接将其迁移以攻击目标黑盒模型。此类攻击的高效性使其对基于ViT的应用构成严重安全威胁，因此，在安全敏感场景中预先设计有效的迁移攻击以识别ViT缺陷至关重要。现有研究通常聚焦于对输入梯度进行正则化以稳定对抗样本的更新方向。然而，ViT中间模块中反向传播梯度的方差可能仍然较大，这可能导致生成的对抗样本聚焦于某些模型特定特征，并陷入较差的局部最优解。为克服现有方法的不足，我们提出了令牌梯度正则化（TGR）方法。根据ViT的结构特性，TGR以令牌为单位降低ViT各内部模块中反向传播梯度的方差，并利用正则化后的梯度生成对抗样本。针对ViT与CNN的广泛攻击实验证实了我们方法的优越性。值得注意的是，与最先进的迁移攻击相比，我们的TGR平均性能提升达8.8%。