We present a novel approach - CLAA - for API aspect detection in API reviews that utilizes transformer models trained with a supervised contrastive loss objective function. We evaluate CLAA using performance and impact analysis. For performance analysis, we utilized a benchmark dataset on developer discussions collected from Stack Overflow and compare the results to those obtained using state-of-the-art transformer models. Our experiments show that contrastive learning can significantly improve the performance of transformer models in detecting aspects such as Performance, Security, Usability, and Documentation. For impact analysis, we performed empirical and developer study. On a randomly selected and manually labeled 200 online reviews, CLAA achieved 92% accuracy while the SOTA baseline achieved 81.5%. According to our developer study involving 10 participants, the use of 'Stack Overflow + CLAA' resulted in increased accuracy and confidence during API selection. Replication package: https://github.com/disa-lab/Contrastive-Learning-API-Aspect-ASE2023
翻译:我们提出了一种名为CLAA的新方法,用于API评论中的API方面检测,该方法利用通过监督对比损失目标函数训练的Transformer模型。我们通过性能分析和影响分析对CLAA进行评估。在性能分析中,我们使用了从Stack Overflow收集的开发者讨论基准数据集,并将结果与使用最先进的Transformer模型获得的结果进行比较。我们的实验表明,对比学习可以显著提高Transformer模型在检测性能、安全性、可用性和文档等维度方面的性能。在影响分析中,我们进行了实证研究和开发者研究。在随机选取并手动标注的200条在线评论中,CLAA达到了92%的准确率,而SOTA基线模型只达到81.5%。根据我们涉及10名参与者的开发者研究,使用"Stack Overflow + CLAA"方法在API选择过程中提高了准确性和置信度。可复现实验包:https://github.com/disa-lab/Contrastive-Learning-API-Aspect-ASE2023