We present a novel approach - CLAA - for API aspect detection in API reviews that utilizes transformer models trained with a supervised contrastive loss objective function. We evaluate CLAA using performance and impact analysis. For performance analysis, we utilized a benchmark dataset on developer discussions collected from Stack Overflow and compare the results to those obtained using state-of-the-art transformer models. Our experiments show that contrastive learning can significantly improve the performance of transformer models in detecting aspects such as Performance, Security, Usability, and Documentation. For impact analysis, we performed empirical and developer study. On a randomly selected and manually labeled 200 online reviews, CLAA achieved 92% accuracy while the SOTA baseline achieved 81.5%. According to our developer study involving 10 participants, the use of 'Stack Overflow + CLAA' resulted in increased accuracy and confidence during API selection. Replication package: https://github.com/shahariar-shibli/Contrastive-Learning-for-API-Aspect-Analysis
翻译:我们提出了一种新颖方法——CLAA,用于API评论中的API方面检测,该方法利用通过监督对比损失目标函数训练的Transformer模型。我们通过性能分析和影响分析评估CLAA。在性能分析中,我们使用了从Stack Overflow收集的开发者讨论基准数据集,并将结果与使用最先进Transformer模型获得的结果进行比较。实验表明,对比学习能显著提升Transformer模型在检测性能、安全性、可用性和文档等属性方面的表现。在影响分析中,我们进行了实证研究和开发者研究。在随机选取并手动标注的200条在线评论上,CLAA达到了92%的准确率,而State-of-the-Art基线模型为81.5%。根据涉及10名参与者的开发者研究,使用'Stack Overflow + CLAA'在API选择过程中提高了准确性和信心。复现包:https://github.com/shahariar-shibli/Contrastive-Learning-for-API-Aspect-Analysis