Context: In the fast-paced evolution of software development, Large Language Models (LLMs) have become indispensable tools for tasks such as code generation, completion, analysis, and bug fixing. Ensuring the robustness of these models against potential vulnerabilities from handling diverse inputs is critical, as variations in input can lead to incorrect or insecure code outputs. Objective: This work aims to improve the robustness of LLMs for coding-related tasks against potential adversarial inputs. Specifically, we investigate how fine-tuning LLMs with perturbed datasets impacts their robustness against input perturbations. Method: We systematically evaluated LLM robustness by fine-tuning models using datasets perturbed at character-level, word-level, and sentence-level, comparing results against base models and models fine-tuned on unperturbed datasets. Results: Fine-tuning LLMs with perturbed datasets significantly improves model robustness (RD usually drops around 4\% - 6\%), especially for models with relatively weak robustness. However, this fine-tuning process typically results in a slight performance decrease (pass@1 usually drops around 1\% - 3\%) compared to fine-tuning with unperturbed datasets, although occasional performance improvements are observed. Conclusion \& Implications: Fine-tuning LLMs for coding tasks with perturbed data effectively enhances their robustness at the cost of a minor performance reduction, emphasizing the importance of balancing the robustness and performance of LLMs for coding applications.

翻译：背景：在快速发展的软件开发领域，大型语言模型（LLMs）已成为代码生成、补全、分析和缺陷修复等任务中不可或缺的工具。由于输入数据的细微变化可能导致错误或不安全的代码输出，确保这些模型在处理多样化输入时抵御潜在漏洞的鲁棒性至关重要。目标：本研究旨在提升LLMs在代码相关任务中对潜在对抗性输入的鲁棒性。具体而言，我们探究了使用扰动数据集微调LLMs如何影响其应对输入扰动的鲁棒性。方法：我们通过使用字符级、词级和句子级扰动数据集对模型进行微调，系统评估了LLMs的鲁棒性，并将结果与基础模型及使用未扰动数据集微调的模型进行对比。结果：使用扰动数据集微调LLMs能显著提升模型鲁棒性（RD通常下降约4%–6%），尤其对于初始鲁棒性相对较弱的模型效果更为明显。然而，与使用未扰动数据集微调相比，该过程通常会导致模型性能轻微下降（pass@1通常降低约1%–3%），尽管偶尔也能观察到性能提升。结论与启示：使用扰动数据对LLMs进行代码任务微调，虽以轻微的性能下降为代价，但能有效增强其鲁棒性，这强调了在代码应用场景中平衡LLMs鲁棒性与性能的重要性。