Context: Annually, ICSE acknowledges a range of papers, a subset of which are paired with research artifacts such as source code, datasets, and supplementary materials, adhering to the Open Science Policy. However, no prior systematic inquiry dives into gauging the influence of ICSE papers using artifact attributes. Objective: We explore the mutual impact between artifacts and their associated papers presented at ICSE over ten years. Method: We collect data on usage attributes from papers and their artifacts, conduct a statistical assessment to identify differences, and analyze the top five papers in each attribute category. Results: There is a significant difference between paper citations and the usage of associated artifacts. While statistical analyses show no notable difference between paper citations and GitHub stars, variations exist in views and/or downloads of papers and artifacts. Conclusion: We provide a thorough overview of ICSE's accepted papers from the last decade, emphasizing the intricate relationship between research papers and their artifacts. To enhance the assessment of artifact influence in software research, we recommend considering key attributes that may be present in one platform but not in another.
翻译:背景:每年,ICSE会表彰一系列论文,其中部分论文遵循开放科学政策,附带了源代码、数据集及补充材料等研究工件。然而,此前尚无系统性研究通过工件属性评估ICSE论文的影响力。目标:我们探究了过去十年间ICSE论文与其工件之间的相互影响。方法:我们收集了论文及其工件的使用属性数据,通过统计评估识别差异,并分析了各属性类别中的前五篇论文。结果:论文引用量与关联工件的使用量之间存在显著差异。虽然统计分析显示论文引用量与GitHub星级之间无显著差异,但论文与工件的浏览量和/或下载量存在差异。结论:我们全面概述了过去十年ICSE已接收论文的情况,重点揭示了研究论文与工件之间的复杂关系。为提升软件研究中工件影响力的评估效果,我们建议纳入那些可能存在于某一平台但缺失于另一平台的关键属性。