Pre-trained code models are mainly evaluated using the in-distribution test data. The robustness of models, i.e., the ability to handle hard unseen data, still lacks evaluation. In this paper, we propose a novel search-based black-box adversarial attack guided by model behaviours for pre-trained programming language models, named Representation Nearest Neighbor Search(RNNS), to evaluate the robustness of Pre-trained PL models. Unlike other black-box adversarial attacks, RNNS uses the model-change signal to guide the search in the space of the variable names collected from real-world projects. Specifically, RNNS contains two main steps, 1) indicate which variable (attack position location) we should attack based on model uncertainty, and 2) search which adversarial tokens we should use for variable renaming according to the model behaviour observations. We evaluate RNNS on 6 code tasks (e.g., clone detection), 3 programming languages (Java, Python, and C), and 3 pre-trained code models: CodeBERT, GraphCodeBERT, and CodeT5. The results demonstrate that RNNS outperforms the state-of-the-art black-box attacking methods (MHM and ALERT) in terms of attack success rate (ASR) and query times (QT). The perturbation of generated adversarial examples from RNNS is smaller than the baselines with respect to the number of replaced variables and the variable length change. Our experiments also show that RNNS is efficient in attacking the defended models and is useful for adversarial training.

翻译：预训练代码模型主要在分布内测试数据上进行评估，但模型的鲁棒性（即处理困难未见数据的能力）仍缺乏评估。本文提出一种基于模型行为引导的新型搜索型黑盒对抗攻击方法——表示最近邻搜索（RNNS），用于评估预训练编程语言模型的鲁棒性。与其他黑盒对抗攻击不同，RNNS利用模型变化信号指导从真实项目中收集的变量名空间的搜索。具体而言，RNNS包含两个主要步骤：1）基于模型不确定性确定需要攻击的变量（攻击位置定位）；2）根据模型行为观测结果搜索用于变量重命名的对抗性令牌。我们在6项代码任务（如克隆检测）、3种编程语言（Java、Python和C）以及3个预训练代码模型（CodeBERT、GraphCodeBERT和CodeT5）上评估了RNNS。结果表明，RNNS在攻击成功率（ASR）和查询次数（QT）上均优于现有最优黑盒攻击方法（MHM和ALERT）。在替换变量数量与变量长度变化方面，RNNS生成的对抗样本扰动小于基线方法。实验还表明，RNNS能有效攻击防御模型，并可用于对抗训练。