Large Language Models (LLMs) have surged as a transformative technology for science and society, prompting governments worldwide to pursue sovereign AI capabilities that ensure data compliance and cultural representation. However, the associated capital costs and engineering complexity required to train these models have largely restricted such capabilities to the private sector, leaving a significant gap for public institutions. This paper details the engineering journey behind training \textit{Apertus}, a fully open multilingual foundation model, on the \textit{Alps} supercomputer. Representing a first-of-its-kind achievement for academia at the 70B parameter scale, we successfully deployed a massive pre-training campaign on one of Europe's largest systems for open science, powered by NVIDIA GH200 Grace Hopper Superchips. We detail the challenges encountered in readying HPC infrastructure for training AI models, from overcoming storage bottlenecks to stabilizing large-scale interconnects, and the lessons learned in transforming a supercomputer into a resilient software-defined Machine Learning Platform. Finally, we discuss the post-training requirements and evolution of our Machine Learning platform, outlining how this initial release lays the groundwork for a sustained, iterative operational capability, in particular for fine tuning foundation models, that extends well beyond a single model training run.

翻译：大型语言模型（LLMs）已成为推动科学与社会变革的变革性技术，促使各国政府追求主权人工智能能力，以确保数据合规性和文化代表性。然而，训练这些模型所需的资本成本与工程复杂性，使其主要局限于私营部门，公共机构因此面临显著的能力缺口。本文详细阐述了在Alps超级计算机上训练全开源多语言基础模型Apertus的工程实践。作为学术界首次在700亿参数规模上取得的里程碑式成就，我们成功在欧洲最大的开放科学系统之一上部署了大规模预训练任务，该系统采用NVIDIA GH200 Grace Hopper超级芯片驱动。我们详细介绍了将高性能计算（HPC）基础设施适配为AI模型训练环境时所面临的挑战——从解决存储瓶颈到稳定大规模互连网络，并总结了将超级计算机转变为弹性软件定义机器学习平台的经验教训。最后，我们讨论了训练后的需求及其对机器学习平台的演进作用，阐明本次初步发布如何为持续迭代的运营能力（尤其针对基础模型微调）奠定基础，其影响远超越单次模型训练本身。