While extensive guidance exists for ensuring the reproducibility of one's own study, there is little discussion regarding the reproduction and replication of external studies within one's own research. To initiate this discussion, drawing lessons from our experience reproducing an operational product for predicting tropical cyclogenesis, we present a two-dimensional framework to offer guidance on reproduction and replication. Our framework, representing model fitting on one axis and its use in inference on the other, builds upon three key aspects: the dataset, the metrics, and the model itself. By assessing the trajectories of our studies on this 2D plane, we can better inform the claims made using our research. Additionally, we use this framework to contextualize the utility of benchmark datasets in the atmospheric sciences. Our two-dimensional framework provides a tool for researchers, especially early career researchers, to incorporate prior work in their own research and to inform the claims they can make in this context.
翻译:摘要:尽管已有大量关于确保自身研究可再现性的指导,但鲜有研究讨论如何在自身研究中对他人研究进行复现与重复。为开启这一讨论,我们基于复现一个用于预测热带气旋生成的业务产品的经验教训,提出一个二维框架以提供复现与重复的指导。该框架在一条轴上表征模型拟合,另一条轴上表征其在推理中的应用,并围绕三个关键方面构建:数据集、评估指标及模型本身。通过评估我们研究在此二维平面上的轨迹,可以更清晰地验证研究结论。此外,我们利用该框架阐明了大气科学中基准数据集的实用价值。该二维框架为研究人员(尤其是早期职业研究者)提供了一种工具,助其将前人工作融入自身研究,并明确在此背景下可推导的结论。