Despite the remarkable achievements of large language models (LLMs) in various tasks, there remains a linguistic bias that favors high-resource languages, such as English, often at the expense of low-resource and regional languages. To address this imbalance, we introduce SeaLLMs, an innovative series of language models that specifically focuses on Southeast Asian (SEA) languages. SeaLLMs are built upon the Llama-2 model and further advanced through continued pre-training with an extended vocabulary, specialized instruction and alignment tuning to better capture the intricacies of regional languages. This allows them to respect and reflect local cultural norms, customs, stylistic preferences, and legal considerations. Our comprehensive evaluation demonstrates that SeaLLM-13b models exhibit superior performance across a wide spectrum of linguistic tasks and assistant-style instruction-following capabilities relative to comparable open-source models. Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai, Khmer, Lao, and Burmese, by large margins while remaining lightweight and cost-effective to operate.

翻译：尽管大语言模型（LLMs）在各种任务中取得了显著成就，但其仍存在语言偏见，往往以牺牲低资源语言和地区性语言为代价，偏向英语等高资源语言。为应对这种不平衡，我们推出了SeaLLMs——一个专注于东南亚语言的创新性语言模型系列。SeaLLMs基于Llama-2模型构建，通过扩展词汇的持续预训练、专业化指令微调与对齐调优，进一步提升了模型对地区语言复杂特性的捕捉能力。这使得模型能够尊重并反映当地的文化规范、风俗习惯、风格偏好及法律考量。我们的综合评估表明，相较于同类开源模型，SeaLLM-13b模型在广泛的语言任务及助手式指令遵循能力上均展现出卓越性能。此外，该模型在泰语、高棉语、老挝语和缅甸语等非拉丁语系语言上的表现大幅超越ChatGPT-3.5，同时保持轻量化与高运行成本效益。