Software defects heavily affect software's functionalities and may cause huge losses. Recently, many AI-based approaches have been proposed to detect defects, which can be divided into two categories: software defect prediction and automatic unit test generation. While these approaches have made great progress in software defect detection, they still have several limitations in practical application, including the low confidence of prediction models and the inefficiency of unit testing models. To address these limitations, we propose a WYSIWYG (i.e., What You See Is What You Get) approach: Attention-based Self-guided Automatic Unit Test GenERation (AUGER), which contains two stages: defect detection and error triggering. In the former stage, AUGER first detects the proneness of defects. Then, in the latter stage, it guides to generate unit tests for triggering such an error with the help of critical information obtained by the former stage. To evaluate the effectiveness of AUGER, we conduct a large-scale experiment by comparing with the state-of-the-art (SOTA) approaches on the widely used datasets (i.e., Bears, Bugs.jar, and Defects4J). AUGER makes great improvements by 4.7% to 35.3% and 17.7% to 40.4% in terms of F1-score and Precision in defect detection, and can trigger 23 to 84 more errors than SOTAs in unit test generation. Besides, we also conduct a further study to verify the generalization in practical usage by collecting a new dataset from real-world projects.
翻译:软件缺陷严重影响软件功能并可能造成巨大损失。近期,众多基于人工智能的方法被提出用于缺陷检测,主要可分为软件缺陷预测与自动单元测试生成两类。尽管这些方法在软件缺陷检测方面取得了显著进展,但在实际应用中仍存在若干局限性,包括预测模型置信度低、单元测试模型效率不足等问题。为克服这些局限,本文提出一种所见即所得(WYSIWYG)方法:基于注意力的自引导自动单元测试生成框架(AUGER)。该框架包含缺陷检测与错误触发两个阶段:在前一阶段,AUGER首先检测缺陷倾向性;随后在后一阶段,利用前一阶段获取的关键信息引导生成触发此类错误的单元测试。为评估AUGER的有效性,我们在广泛使用的数据集(Bears、Bugs.jar和Defects4J)上进行了大规模实验,并与前沿方法进行对比。实验表明,AUGER在缺陷检测的F1分数和精确率上分别提升了4.7%至35.3%与17.7%至40.4%,在单元测试生成中比前沿方法多触发23至84个错误。此外,我们还通过从实际项目中收集新数据集进行了深入研究,验证了该方法在实际应用中的泛化能力。