The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard MLOps tools and platforms, and the unique requirements of modern and Open Science, particularly regarding the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. This paper presents AI4EOSC, a federated, open-source platform designed to operationalize the full AI/ML lifecycle within the European Open Science Cloud (EOSC) ecosystem. Our methodology tackles the fragmentation of distributed research infrastructures by integrating a modular and distributed architecture comprising an AI development platform, a serverless AI-as-a-Service layer, and a federated orchestration model that is able to integrate heterogeneous compute and storage resources from distributed e-Infrastructures. AI4EOSC also introduces a ``FAIR-by-design'' approach that enforces metadata standardization (via MLDCAT-AP) and W3C PROV-compliant provenance tracking through a platform-integrated CI/CD pipeline. AI4EOSC added value is demonstrated through the delivery of a diverse set of community installations, showing consistent and seamless deployment across heterogeneous cloud providers. These installations are validated by a set of scientific cases, showing how our work reduces the manual burden on researchers while ensuring high levels of reproducibility and interoperability and providing an unified environment for development, training, and production of AI/ML models in the EOSC.
翻译:人工智能与机器学习在科学研究中的迅猛发展,凸显了行业标准的MLOps工具与平台,同现代开放科学(尤其是FAIR原则——可发现、可访问、可互操作、可复用)的特殊需求之间的差距。本文提出AI4EOSC——一个联邦式开源平台,旨在欧洲开放科学云(EOSC)生态系统中实现完整AI/ML生命周期的运行。我们的方法通过集成模块化分布式架构来应对分布式研究基础设施的碎片化问题,该架构包含AI开发平台、无服务器AI即服务层,以及能够整合来自分布式电子基础设施的异构计算与存储资源的联邦编排模型。AI4EOSC还引入了"FAIR-by-design"方法,通过平台集成的CI/CD流水线强制执行元数据标准化(基于MLDCAT-AP)与符合W3C PROV标准的溯源追踪。AI4EOSC的价值通过交付一系列多样化的社区部署得到验证,展现了其在异构云提供商间一致且无缝的部署能力。这些部署经一系列科学案例验证,表明我们的工作如何减少研究人员的手动负担,同时确保高水平的可复现性与互操作性,并为EOSC中AI/ML模型的开发、训练与生产提供统一环境。