Despite the remarkable achievements of large language models (LLMs) in various tasks, there remains a linguistic bias that favors high-resource languages, such as English, often at the expense of low-resource and regional languages. To address this imbalance, we introduce SeaLLMs, an innovative series of language models that specifically focuses on Southeast Asian (SEA) languages. SeaLLMs are built upon the Llama-2 model and further advanced through continued pre-training with an extended vocabulary, specialized instruction and alignment tuning to better capture the intricacies of regional languages. This allows them to respect and reflect local cultural norms, customs, stylistic preferences, and legal considerations. Our comprehensive evaluation demonstrates that SeaLLM-13b models exhibit superior performance across a wide spectrum of linguistic tasks and assistant-style instruction-following capabilities relative to comparable open-source models. Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai, Khmer, Lao, and Burmese, by large margins while remaining lightweight and cost-effective to operate.

翻译：尽管大型语言模型（LLM）在各类任务中取得了显著成就，但仍存在语言偏见问题，即偏向英语等高资源语言，而往往牺牲低资源语言和区域语言。为应对这一失衡，我们推出SeaLLMs，这是一系列专注于东南亚（SEA）语言的创新语言模型。SeaLLMs基于Llama-2模型构建，通过持续预训练扩展词汇表，并借助专门的指令调优和对齐调优，更好地捕捉区域语言的细微差异。这使得它们能够尊重并反映当地文化规范、习俗、风格偏好及法律考量。我们的综合评估表明，SeaLLM-13b模型在广泛的语言任务和助手式指令遵循能力方面，相较于同类开源模型展现出优越性能。此外，在泰语、高棉语、老挝语和缅甸语等非拉丁语言中，SeaLLM-13b以轻量化和低成本运行的显著优势，大幅超越了ChatGPT-3.5。