Today, the security of many domains rely on the use of Machine Learning to detect threats, identify vulnerabilities, and safeguard systems from attacks. Recently, transformer architectures have improved the state-of-the-art performance on a wide range of tasks such as malware detection and network intrusion detection. But, before abandoning current approaches to transformers, it is crucial to understand their properties and implications on cybersecurity applications. In this paper, we evaluate the robustness of transformers to adversarial samples for system defenders (i.e., resiliency to adversarial perturbations generated on different types of architectures) and their adversarial strength for system attackers (i.e., transferability of adversarial samples generated by transformers to other target models). To that effect, we first fine-tune a set of pre-trained transformer, Convolutional Neural Network (CNN), and hybrid (an ensemble of transformer and CNN) models to solve different downstream image-based tasks. Then, we use an attack algorithm to craft 19,367 adversarial examples on each model for each task. The transferability of these adversarial examples is measured by evaluating each set on other models to determine which models offer more adversarial strength, and consequently, more robustness against these attacks. We find that the adversarial examples crafted on transformers offer the highest transferability rate (i.e., 25.7% higher than the average) onto other models. Similarly, adversarial examples crafted on other models have the lowest rate of transferability (i.e., 56.7% lower than the average) onto transformers. Our work emphasizes the importance of studying transformer architectures for attacking and defending models in security domains, and suggests using them as the primary architecture in transfer attack settings.

翻译：如今，众多安全领域依赖机器学习来检测威胁、识别漏洞并防范系统攻击。近年来，Transformer架构在恶意软件检测和网络入侵检测等广泛任务中提升了现有最优性能。但在全面采用Transformer替代当前方法之前，理解其特性及对网络安全应用的影响至关重要。本文从系统防御者角度评估Transformer对对抗样本的鲁棒性（即对不同架构生成对抗扰动的恢复能力），以及从系统攻击者角度评估其对抗强度（即Transformer生成的对抗样本向其他目标模型的迁移能力）。为此，我们首先微调一组预训练的Transformer、卷积神经网络及混合模型（Transformer与CNN的集成），以解决不同的下游图像任务。随后，针对每个任务，我们使用攻击算法为每个模型生成19,367个对抗样本。通过评估各模型生成的对抗样本在其他模型上的迁移性能，量化迁移率，从而确定哪些模型具有更强的对抗攻击强度及更高的对抗鲁棒性。实验发现，基于Transformer生成的对抗样本向其他模型的迁移率最高（比平均值高25.7%）；反之，其他模型生成的对抗样本向Transformer的迁移率最低（比平均值低56.7%）。本研究强调了在安全领域攻防模型中研究Transformer架构的重要性，并建议在迁移攻击场景中将其作为主要架构。