The increasing complexity of today's software requires the contribution of thousands of developers. This complex collaboration structure makes developers more likely to introduce defect-prone changes that lead to software faults. Determining when these defect-prone changes are introduced has proven challenging, and using traditional machine learning (ML) methods to make these determinations seems to have reached a plateau. In this work, we build contribution graphs consisting of developers and source files to capture the nuanced complexity of changes required to build software. By leveraging these contribution graphs, our research shows the potential of using graph-based ML to improve Just-In-Time (JIT) defect prediction. We hypothesize that features extracted from the contribution graphs may be better predictors of defect-prone changes than intrinsic features derived from software characteristics. We corroborate our hypothesis using graph-based ML for classifying edges that represent defect-prone changes. This new framing of the JIT defect prediction problem leads to remarkably better results. We test our approach on 14 open-source projects and show that our best model can predict whether or not a code change will lead to a defect with an F1 score as high as 77.55% and a Matthews correlation coefficient (MCC) as high as 53.16%. This represents a 152% higher F1 score and a 3% higher MCC over the state-of-the-art JIT defect prediction. We describe limitations, open challenges, and how this method can be used for operational JIT defect prediction.
翻译:当今软件的日益复杂性需要数千名开发者的贡献。这种复杂的协作结构使得开发者更有可能引入容易导致软件缺陷的变更。确定这些易缺陷变更何时被引入已被证明具有挑战性,而使用传统机器学习方法进行此类判断似乎已遇到瓶颈。在本工作中,我们构建了由开发者和源文件组成的贡献图,以捕捉构建软件所需变更的细微复杂性。通过利用这些贡献图,我们的研究展示了基于图的机器学习在改进即时缺陷预测方面的潜力。我们假设从贡献图中提取的特征比从软件特性派生出的内在特征更能预测易缺陷变更。我们使用基于图的机器学习对表示易缺陷变更的边进行分类,从而验证了这一假设。这种对即时缺陷预测问题的新颖表述带来了显著更好的结果。我们在14个开源项目上测试了我们的方法,并表明最佳模型能够以高达77.55%的F1分数和高达53.16%的马修斯相关系数来预测代码变更是否会导致缺陷。相较于最先进的即时缺陷预测方法,这表示F1分数提高了152%,马修斯相关系数提高了3%。我们描述了局限性、开放挑战,以及该方法如何用于实际的即时缺陷预测。