Large language models demonstrate remarkable proficiency in various linguistic tasks and have extensive knowledge across various domains. Although they perform best in English, their ability in other languages is notable too. In contrast, open-source models, such as LLaMa, are primarily trained on English datasets, resulting in poor performance in non-English languages. In this paper, we introduce PersianMind, an open-source bilingual large language model which demonstrates comparable performance to closed-source GPT-3.5-turbo in the Persian language. By expanding LLaMa2's vocabulary with 10,000 Persian tokens and training it on a dataset comprising nearly 2 billion Persian tokens, we show that our approach preserves the model's English knowledge and employs transfer learning to excel at transferring task knowledge from one language to another.

翻译：大型语言模型在多种语言任务中展现出卓越的能力，并拥有跨领域广泛的知识。尽管它们在英语中表现最佳，但在其他语言中的能力同样显著。相比之下，诸如LLaMa等开源模型主要基于英语数据集训练，导致其在非英语语言中性能较差。本文提出了PersianMind，一种开源双语大型语言模型，其在波斯语中的表现与闭源的GPT-3.5-turbo不相上下。通过将LLaMa2的词汇表扩展10,000个波斯语标记，并使用包含近20亿波斯语标记的数据集进行训练，我们证明了该方法能保留模型的英语知识，并利用迁移学习在语言间高效传递任务知识。