Pre-trained Code Language Models (Code-PLMs) have shown many advancements and achieved state-of-the-art results for many software engineering tasks in the past few years. These models are mainly targeted for popular programming languages such as Java and Python, leaving out many other ones like R. Though R has a wide community of developers and users, there is little known about the applicability of Code-PLMs for R. In this preliminary study, we aim to investigate the vulnerability of Code-PLMs for code entities in R. For this purpose, we use an R dataset of code and comment pairs and then apply CodeAttack, a black-box attack model that uses the structure of code to generate adversarial code samples. We investigate how the model can attack different entities in R. This is the first step towards understanding the importance of R token types, compared to popular programming languages (e.g., Java). We limit our study to code summarization. Our results show that the most vulnerable code entity is the identifier, followed by some syntax tokens specific to R. The results can shed light on the importance of token types and help in developing models for code summarization and method name prediction for the R language.

翻译：预训练代码语言模型（Code-PLMs）近年来在多项软件工程任务中展现出显著进展，并取得了最先进的成果。这些模型主要针对Java和Python等主流编程语言，而忽略了R等其他语言。尽管R拥有庞大的开发者与用户社区，但Code-PLMs在R语言上的适用性研究尚属空白。在本项初步研究中，我们旨在探究Code-PLMs对R语言中代码实体的脆弱性。为此，我们采用一个包含代码与注释对的R语言数据集，并应用CodeAttack（一种利用代码结构生成对抗性代码样本的黑盒攻击模型）进行分析。我们研究了该模型对R语言中不同实体进行攻击的机制。这是理解R语言令牌类型重要性的第一步（相较于Java等主流编程语言）。我们将研究范围限定于代码摘要任务。实验结果表明，最脆弱的代码实体是标识符，其次是一些R语言特有的语法令牌。本研究成果可揭示令牌类型的重要性，并为开发面向R语言的代码摘要及方法名预测模型提供指导。