API misuses often lead to software bugs, crashes, and vulnerabilities. While several API misuse detectors have been proposed, there are no automatic repair tools specifically designed for this purpose. In a recent study, test-suite-based automatic program repair (APR) tools were found to be ineffective in repairing API misuses. Still, since the study focused on non-learning-aided APR tools, it remains unknown whether learning-aided APR tools are capable of fixing API misuses. In recent years, pre-trained language models (PLMs) have succeeded greatly in many natural language processing tasks. There is a rising interest in applying PLMs to APR. However, there has not been any study that investigates the effectiveness of PLMs in repairing API misuse. To fill this gap, we conduct a comprehensive empirical study on 11 learning-aided APR tools, which include 9 of the state-of-the-art general-purpose PLMs and two APR tools. We evaluate these models with an API-misuse repair dataset, consisting of two variants. Our results show that PLMs perform better than the studied APR tools in repairing API misuses. Among the 9 pre-trained models tested, CodeT5 is the best performer in the exact match. We also offer insights and potential exploration directions for future research.
翻译:API误用常导致软件缺陷、崩溃和漏洞。尽管已有多种API误用检测工具被提出,但目前尚无专门针对此问题的自动修复工具。近期研究表明,基于测试套件的自动程序修复(APR)工具在修复API误用方面效果不佳。然而,由于该研究聚焦于非学习辅助型APR工具,学习辅助型APR工具能否修复API误用仍属未知。近年来,预训练语言模型(PLMs)在众多自然语言处理任务中取得了巨大成功,将PLMs应用于APR的研究兴趣日益增长。但尚无研究探讨PLMs修复API误用的有效性。为填补这一空白,我们对11种学习辅助型APR工具开展了全面实证研究,其中包括9种最先进的通用PLMs和两种APR工具。我们使用包含两种变体的API误用修复数据集评估这些模型。结果表明,PLMs在修复API误用方面的表现优于所研究的APR工具。在测试的9种预训练模型中,CodeT5在精确匹配指标上表现最佳。我们还为未来研究提供了洞见和潜在的探索方向。