Modern Deep Learning (DL) architectures based on transformers (e.g., BERT, RoBERTa) are exhibiting performance improvements across a number of natural language tasks. While such DL models have shown tremendous potential for use in software engineering applications, they are often hampered by insufficient training data. Particularly constrained are applications that require project-specific data, such as bug localization, which aims at recommending code to fix a newly submitted bug report. Deep learning models for bug localization require a substantial training set of fixed bug reports, which are at a limited quantity even in popular and actively developed software projects. In this paper, we examine the effect of using synthetic training data on transformer-based DL models that perform a more complex variant of bug localization, which has the goal of retrieving bug-inducing changesets for each bug report. To generate high-quality synthetic data, we propose novel data augmentation operators that act on different constituent components of bug reports. We also describe a data balancing strategy that aims to create a corpus of augmented bug reports that better reflects the entire source code base, because existing bug reports used as training data usually reference a small part of the code base.

翻译：现代基于Transformer的深度学习架构（如BERT、RoBERTa）在多项自然语言任务中展现出性能提升。尽管此类深度学习模型在软件工程应用中表现出巨大潜力，但其常受限于训练数据不足。尤其受制约的是需要项目特定数据的应用场景，例如缺陷定位——旨在为新提交的缺陷报告推荐修复代码。用于缺陷定位的深度学习模型需要大量已修复缺陷报告的训练集，即便在流行且积极开发的软件项目中，此类报告的数量也十分有限。本文研究了使用合成训练数据对执行更复杂缺陷定位变体（旨在为每个缺陷报告检索引发缺陷的变更集）的Transformer深度学习模型的影响。为生成高质量合成数据，我们提出了作用于缺陷报告不同组成成分的新型数据增强算子。我们还描述了一种数据平衡策略，旨在构建能更好反映整个源代码库的增强缺陷报告语料库，因为用作训练数据的现有缺陷报告通常仅引用代码库的少部分内容。