Software bug reports reported on bug-tracking systems often lack crucial information for the developers to promptly resolve them, costing companies billions of dollars. There has been significant research on effectively eliciting information from bug reporters in bug tracking systems using different templates that bug reporters need to use. However, the need for asking follow-up questions persists. Recent studies propose techniques to suggest these follow-up questions to help developers obtain the missing details, but there has been little research on answering these follow up questions, which are often unanswered. In this paper, we propose a novel approach that uses CodeT5 in combination with Lucene, an information retrieval technique that leverages the relevance of different bug reports, their components, and follow-up questions to recommend answers. These top-performing answers, along with their bug report, serve as additional context apart from the deficient bug report to the deep learning model for generating an answer. We evaluate our recommended answers with the manually annotated answers using similarity metrics like Normalized Smooth BLEU Score, METEOR, Word Mover's Distance, and Semantic Similarity. We achieve a BLEU Score of up to 34 and Semantic Similarity of up to 64 which shows that the answers generated are understandable and good according to Google's standard and can outperform multiple baselines.
翻译:摘要:软件缺陷跟踪系统中提交的缺陷报告通常缺乏关键信息,导致开发人员无法及时解决问题,给企业造成数十亿美元的损失。已有大量研究通过要求缺陷报告者使用不同模板来有效获取信息。然而,提出后续问题的需求依然存在。近期研究提出了推荐这些后续问题的技术,以帮助开发人员获取缺失细节,但关于回答这些常被忽略的后续问题的研究尚不充分。本文提出了一种新颖方法,结合CodeT5与Lucene——一种利用不同缺陷报告及其组件和后续问题相关性来推荐答案的信息检索技术。除原有缺陷报告外,这些高性能答案及其对应缺陷报告将作为额外上下文输入深度学习模型以生成答案。我们使用归一化平滑BLEU分数、METEOR、词移距离和语义相似度等指标,将推荐答案与人工标注答案进行对比评估。结果表明,生成的答案在BLEU分数(最高达34)和语义相似度(最高达64)方面均表现良好,符合谷歌标准,并能超越多个基线模型。