Defect prediction aims at identifying software components that are likely to cause faults before a software is made available to the end-user. To date, this task has been modeled as a two-class classification problem, however its nature also allows it to be formulated as a one-class classification task. Previous studies show that One-Class Support Vector Machine (OCSVM) can outperform two-class classifiers for within-project defect prediction, however it is not effective when employed at a finer granularity (i.e., commit-level defect prediction). In this paper, we further investigate whether learning from one class only is sufficient to produce effective defect prediction model in two other different scenarios (i.e., granularity), namely cross-version and cross-project defect prediction models, as well as replicate the previous work at within-project granularity for completeness. Our empirical results confirm that OCSVM performance remain low at different granularity levels, that is, it is outperformed by the two-class Random Forest (RF) classifier for both cross-version and cross-project defect prediction. While, we cannot conclude that OCSVM is the best classifier, our results still show interesting findings. While OCSVM does not outperform RF, it still achieves performance superior to its two-class counterpart (i.e., SVM) as well as other two-class classifiers studied herein. We also observe that OCSVM is more suitable for both cross-version and cross-project defect prediction, rather than for within-project defect prediction, thus suggesting it performs better with heterogeneous data. We encourage further research on one-class classifiers for defect prediction as these techniques may serve as an alternative when data about defective modules is scarce or not available.
翻译:缺陷预测旨在软件交付最终用户前,识别可能引发故障的软件组件。目前该任务通常被建模为二分类问题,但其本质也可被表述为一类分类任务。先前研究表明,一类支持向量机(OCSVM)在项目内缺陷预测中可优于二分类器,但在更细粒度(即提交级缺陷预测)场景中效果不佳。本文进一步探究仅通过单类学习是否足以在另外两种不同场景(即跨版本与跨项目缺陷预测模型)中构建有效的缺陷预测模型,同时为完整性复现先前工作,验证项目内粒度下的表现。实证结果表明,OCSVM在不同粒度层级上性能均较低——在跨版本与跨项目缺陷预测中均被二分类随机森林(RF)分类器超越。虽然无法认定OCSVM为最优分类器,但结果仍呈现有趣发现:尽管OCSVM未超越RF,但其性能仍优于对应的二分类器(即SVM)及本文研究的其他二分类器。我们同时观察到,OCSVM更适用于跨版本与跨项目缺陷预测,而非项目内预测,这表明其在异质性数据上表现更佳。我们鼓励进一步开展基于单类分类器的缺陷预测研究,因为当缺陷模块数据稀疏或不可用时,此类技术可作为有效替代方案。