Pre-trained code models are mainly evaluated using the in-distribution test data. The robustness of models, i.e., the ability to handle hard unseen data, still lacks evaluation. In this paper, we propose a novel search-based black-box adversarial attack guided by model behaviours for pre-trained programming language models, named Representation Nearest Neighbor Search(RNNS), to evaluate the robustness of Pre-trained PL models. Unlike other black-box adversarial attacks, RNNS uses the model-change signal to guide the search in the space of the variable names collected from real-world projects. Specifically, RNNS contains two main steps, 1) indicate which variable (attack position location) we should attack based on model uncertainty, and 2) search which adversarial tokens we should use for variable renaming according to the model behaviour observations. We evaluate RNNS on 6 code tasks (e.g., clone detection), 3 programming languages (Java, Python, and C), and 3 pre-trained code models: CodeBERT, GraphCodeBERT, and CodeT5. The results demonstrate that RNNS outperforms the state-of-the-art black-box attacking methods (MHM and ALERT) in terms of attack success rate (ASR) and query times (QT). The perturbation of generated adversarial examples from RNNS is smaller than the baselines with respect to the number of replaced variables and the variable length change. Our experiments also show that RNNS is efficient in attacking the defended models and is useful for adversarial training.

翻译：预训练代码模型主要使用分布内测试数据进行评估，但模型的鲁棒性——即处理困难未见数据的能力——仍缺乏系统性评估。本文提出一种基于模型行为引导的新型搜索式黑盒对抗攻击方法——表征最近邻搜索（RNNS），用于评估预训练编程语言模型的鲁棒性。与现有黑盒对抗攻击不同，RNNS利用模型变化信号指导从真实项目收集的变量名空间中的搜索。具体而言，RNNS包含两个核心步骤：1）基于模型不确定性确定应攻击的变量（攻击位置定位）；2）根据模型行为观测搜索适用于变量重命名的对抗性词元。我们在6项代码任务（如克隆检测）、3种编程语言（Java、Python和C）及3个预训练代码模型（CodeBERT、GraphCodeBERT和CodeT5）上评估了RNNS。结果表明，RNNS在攻击成功率（ASR）和查询次数（QT）上均优于现有最先进的黑盒攻击方法（MHM和ALERT）。相较于基线方法，RNNS生成的对抗样本在替换变量数量和变量长度变化维度上的扰动更小。实验还证明RNNS能高效攻击经防御训练的模型，并对对抗训练具有促进作用。