Norwegian, spoken by approximately five million people, remains underrepresented in many of the most significant breakthroughs in Natural Language Processing (NLP). To address this gap, the NorLLM team at NorwAI has developed a family of models specifically tailored to Norwegian and other Scandinavian languages, building on diverse Transformer-based architectures such as GPT, Mistral, Llama2, Mixtral and Magistral. These models are either pretrained from scratch or continually pretrained on 25B - 88.45B tokens, using a Norwegian-extended tokenizer and advanced post-training strategies to optimize performance, enhance robustness, and improve adaptability across various real-world tasks. Notably, instruction-tuned variants (e.g., Mistral-7B-Instruct and Mixtral-8x7B-Instruct) showcase strong assistant-style capabilities, underscoring their potential for practical deployment in interactive and domain-specific applications. The NorwAI large language models are openly available to Nordic organizations, companies and students for both research and experimental use. This report provides detailed documentation of the model architectures, training data, tokenizer design, fine-tuning strategies, deployment, and evaluations.

翻译：挪威语作为约五百万人口使用的语言，在自然语言处理（NLP）领域的许多重大突破中仍处于代表性不足的状态。为弥补这一差距，NorwAI的NorLLM团队基于GPT、Mistral、Llama2、Mixtral和Magistral等多种Transformer架构，开发了专门针对挪威语及其他斯堪的纳维亚语言的系列模型。这些模型通过使用挪威语扩展的分词器，在250亿至884.5亿词元规模上进行了从头预训练或持续预训练，并采用先进的训练后策略以优化性能、增强鲁棒性，提升其在各类实际任务中的适应性。值得注意的是，经过指令微调的变体（如Mistral-7B-Instruct和Mixtral-8x7B-Instruct）展现出强大的助手式能力，凸显了其在交互式及特定领域应用中实际部署的潜力。NorwAI大型语言模型已向北欧地区的组织、企业和学生开放，供研究及实验使用。本报告详细记录了模型架构、训练数据、分词器设计、微调策略、部署方案及评估结果。