Software developers often submit questions to technical Q&A sites like Stack Overflow (SO) to resolve code-level problems. In practice, they include example code snippets with questions to explain the programming issues. Existing research suggests that users attempt to reproduce the reported issues using given code snippets when answering questions. Unfortunately, such code snippets could not always reproduce the issues due to several unmet challenges that prevent questions from receiving appropriate and prompt solutions. One previous study investigated reproducibility challenges and produced a catalog. However, how the practitioners perceive this challenge catalog is unknown. Practitioners' perspectives are inevitable in validating these challenges and estimating their severity. This study first surveyed 53 practitioners to understand their perspectives on reproducibility challenges. We attempt to (a) see whether they agree with these challenges, (b) determine the impact of each challenge on answering questions, and (c) identify the need for tools to promote reproducibility. Survey results show that - (a) about 90% of the participants agree with the challenges, (b) "missing an important part of code" most severely hurt reproducibility, and (c) participants strongly recommend introducing automated tool support to promote reproducibility. Second, we extract \emph{nine} code-based features (e.g., LOC, compilability) and build five Machine Learning (ML) models to predict issue reproducibility. Early detection might help users improve code snippets and their reproducibility. Our models achieve 84.5% precision, 83.0% recall, 82.8% F1-score, and 82.8% overall accuracy, which are highly promising. Third, we systematically interpret the ML model and explain how code snippets with reproducible issues differ from those with irreproducible issues.
翻译:软件开发人员经常在 Stack Overflow(SO)等技术问答网站上提交问题以解决代码层面的问题。实践中,他们会在问题中包含示例代码片段来解释编程问题。现有研究表明,用户在回答问题时,会尝试使用给定的代码片段复现所报告的问题。遗憾的是,由于一些未满足的挑战阻碍了问题获得适当且及时的解决方案,此类代码片段并不总能复现问题。先前的一项研究调查了可复现性挑战并编制了一个目录。然而,从业者如何看待这个挑战目录尚不清楚。从业者的视角对于验证这些挑战并评估其严重性至关重要。本研究首先调查了 53 位从业者,以了解他们对可复现性挑战的看法。我们试图(a)了解他们是否同意这些挑战,(b)确定每个挑战对回答问题的影响,以及(c)识别是否需要工具来促进可复现性。调查结果显示:(a)约 90% 的参与者同意这些挑战;(b)“缺少代码的重要部分”对可复现性的损害最为严重;(c)参与者强烈建议引入自动化工具支持以促进可复现性。其次,我们提取了九项基于代码的特征(例如,代码行数、可编译性),并构建了五个机器学习模型来预测问题的可复现性。早期检测可能有助于用户改进代码片段及其可复现性。我们的模型达到了 84.5% 的精确率、83.0% 的召回率、82.8% 的 F1 分数和 82.8% 的总体准确率,结果非常有前景。第三,我们系统地解释了机器学习模型,并说明了具有可复现问题的代码片段与具有不可复现问题的代码片段有何不同。