Background and Objectives: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. Methods: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. Results: The AUC values of different runs of the same experiment were generally consistent. However, we observed small variations in AUC values of up to 0.045, indicating a practical limit to reproducibility. Conclusions: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

翻译：背景与目的：可重复性是计算病理学中开发基于机器学习解决方案的主要挑战。NCI影像数据共享库依据FAIR原则提供120余组癌症影像数据集，并设计为可与云机器学习服务协同使用。本研究旨在探索该平台促进计算病理学研究可重复性的潜力。方法：通过IDC实施两项实验，分别在不同数据集上训练和/或评估具有代表性的基于机器学习的肺肿瘤组织分类方法。为评估可重复性，实验在通用机器学习服务的独立但配置完全相同的实例上重复运行。结果：相同实验的多次运行AUC值整体一致，但观察到最大0.045的微小差异，表明存在实际可重复性极限。结论：我们认为IDC通过允许研究者重复使用完全相同的数据集，以及与云机器学习服务的集成，使实验可在完全相同的计算环境中运行，从而推动计算病理学研究接近可重复性极限。