Context: Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transfer-ring knowledge among researchers to facilitate experiment replications. Objective: This paper investigates the experiment replication process to find out what information is needed to successfully replicate an experiment. Our objective is to propose the content and structure of laboratory packages for software engineering experiments. Method: We evaluated seven replications of three different families of experiments. Each replication had a different experimenter who was, at the time, unfamiliar with the experi-ment. During the first iterations of the study, we identified experimental incidents and then proposed a laboratory package structure that addressed these incidents, including docu-ment usability improvements. We used the later iterations to validate and generalize the laboratory package structure for use in all software engineering experiments. We aimed to solve a specific problem, while at the same time looking at how to contribute to the body of knowledge on laboratory packages. Results: We generated a laboratory package for three different experiments. These packages eased the replication of the respective experiments. The evaluation that we conducted shows that the laboratory package proposal is acceptable and reduces the effort currently required to replicate experiments in software engineering. Conclusion: We think that the content and structure that we propose for laboratory pack-ages can be useful for other software engineering experiments.
翻译:背景:实验复现是科学方法的核心环节。尽管软件工程实验已取得长足发展,但实验复现的数量仍然相对较少。软件工程实验包含复杂的概念、流程及工件,而实验室包作为研究者之间传递知识的媒介,旨在促进实验复现。目标:本文旨在探究实验复现过程,明确成功复现实验所需的信息,提出适用于软件工程实验的实验室包内容与结构。方法:我们评估了三个不同实验家族的七组复现实验,每组复现均由当时对实验不熟悉的研究者独立完成。在研究初期迭代中,我们识别了实验过程中的异常事件,并提出包含文档可用性改进的实验室包结构以应对这些问题。后续迭代用于验证并推广该实验室包结构,使其适用于所有软件工程实验。研究在解决特定问题的同时,探索如何丰富实验室包领域的知识体系。结果:我们为三个不同实验生成了实验室包,这些包有效简化了相应实验的复现流程。评估结果显示,所提出的实验室包方案具有可行性,并能显著降低当前软件工程实验复现所需的工作量。结论:我们认为提出的实验室包内容与结构可推广至其他软件工程实验。