While extensive guidance exists for ensuring the reproducibility of one's own study, there is little discussion regarding the reproduction and replication of external studies within one's own research. To initiate this discussion, drawing lessons from our experience reproducing an operational product for predicting tropical cyclogenesis, we present a two-dimensional framework to offer guidance on reproduction and replication. Our framework, representing model fitting on one axis and its use in inference on the other, builds upon three key aspects: the dataset, the metrics, and the model itself. By assessing the trajectories of our studies on this 2D plane, we can better inform the claims made using our research. Additionally, we use this framework to contextualize the utility of benchmark datasets in the atmospheric sciences. Our two-dimensional framework provides a tool for researchers, especially early career researchers, to incorporate prior work in their own research and to inform the claims they can make in this context.
翻译:尽管已有大量指南确保研究的可复现性,但关于如何在自身研究中复现或重复外部研究的讨论却寥寥无几。为开启这一讨论,我们借鉴复现热带气旋生成预测业务产品的经验,提出一个二维框架以指导复现与重复研究。该框架以模型拟合为横轴、推断应用为纵轴,围绕数据集、评估指标及模型本身三个关键维度构建。通过在此二维平面上评估研究轨迹,我们得以更清晰地论证基于研究的结论。同时,我们利用该框架梳理大气科学中基准数据集的实际效用。这一二维框架为研究者(尤其是青年科研人员)提供了工具,使其能在自身研究中整合前人工作,并在此基础上形成可论证的论断。