We introduce Bielik 7B v0.1, a 7-billion-parameter generative text model for Polish language processing. Trained on curated Polish corpora, this model addresses key challenges in language model development through innovative techniques. These include Weighted Instruction Cross-Entropy Loss, which balances the learning of different instruction types, and Adaptive Learning Rate, which dynamically adjusts the learning rate based on training progress. To evaluate performance, we created the Open PL LLM Leaderboard and Polish MT-Bench, novel frameworks assessing various NLP tasks and conversational abilities. Bielik 7B v0.1 demonstrates significant improvements, achieving a 9 percentage point increase in average score compared to Mistral-7B-v0.1 on the RAG Reader task. It also excels in the Polish MT-Bench, particularly in Reasoning (6.15/10) and Role-playing (7.83/10) categories. This model represents a substantial advancement in Polish language AI, offering a powerful tool for diverse linguistic applications and setting new benchmarks in the field.

翻译：我们介绍了Bielik 7B v0.1，这是一个用于波兰语处理的70亿参数生成式文本模型。该模型在精心策划的波兰语语料库上进行训练，通过创新技术解决了语言模型开发中的关键挑战。这些技术包括加权指令交叉熵损失，它平衡了不同指令类型的学习；以及自适应学习率，它根据训练进度动态调整学习率。为了评估性能，我们创建了Open PL LLM排行榜和Polish MT-Bench，这是评估各种自然语言处理任务和对话能力的新型框架。Bielik 7B v0.1展现出显著的改进，在RAG Reader任务上的平均得分相比Mistral-7B-v0.1提高了9个百分点。它在Polish MT-Bench中也表现出色，特别是在推理（6.15/10）和角色扮演（7.83/10）类别中。该模型代表了波兰语人工智能领域的重大进展，为多样化的语言应用提供了强大工具，并为该领域设定了新的基准。