Scientific research is increasingly reliant on computational methods, posing challenges for ensuring research reproducibility. This study focuses on the field of artificial intelligence (AI) and introduces a new framework for evaluating AI platforms for reproducibility from a cyber security standpoint to address the security challenges associated with AI research. Using this framework, five popular AI reproducibility platforms; Floydhub, BEAT, Codalab, Kaggle, and OpenML were assessed. The analysis revealed that none of these platforms fully incorporates the necessary cyber security measures essential for robust reproducibility. Kaggle and Codalab, however, performed better in terms of implementing cyber security measures covering aspects like security, privacy, usability, and trust. Consequently, the study provides tailored recommendations for different user scenarios, including individual researchers, small laboratories, and large corporations. It emphasizes the importance of integrating specific cyber security features into AI platforms to address the challenges associated with AI reproducibility, ultimately advancing reproducibility in this field. Moreover, the proposed framework can be applied beyond AI platforms, serving as a versatile tool for evaluating a wide range of systems and applications from a cyber security perspective.
翻译:科学研究日益依赖计算方法,这给确保研究可复现性带来了挑战。本研究聚焦人工智能领域,从网络安全视角提出了一套评估AI平台可复现性的新框架,以应对AI研究中的安全挑战。利用该框架,对Floydhub、BEAT、Codalab、Kaggle和OpenML这五个主流AI可复现性平台进行了评估。分析表明,这些平台均未完全集成实现稳健可复现性所必需的网络安全措施。但Kaggle和Codalab在实施涵盖安全、隐私、可用性和信任等维度的网络安全措施方面表现更优。据此,研究针对不同用户场景(包括独立研究者、小型实验室和大型企业)提出了定制化建议,强调了将特定网络安全功能整合到AI平台中以应对AI可复现性挑战的重要性,从而推动该领域的可复现性发展。此外,所提出的框架可应用于AI平台之外,作为从网络安全视角评估各类系统和应用的通用工具。