In the realm of language models, the nuanced linguistic and cultural intricacies of Traditional Chinese, as spoken in Taiwan, have been largely overlooked. This paper introduces Taiwan LLM, a pioneering Large Language Model that specifically caters to the Traditional Chinese language, with a focus on the variant used in Taiwan. Leveraging a comprehensive pretraining corpus and instruction-finetuning datasets, we have developed a model that not only understands the complexities of Traditional Chinese but also embodies the cultural context of Taiwan. Taiwan LLM represents the first of its kind, a model that is not only linguistically accurate but also culturally resonant with its user base. Our evaluations demonstrate that Taiwan LLM achieves superior performance in understanding and generating Traditional Chinese text, outperforming existing models that are predominantly trained on Simplified Chinese or English. The open-source release of Taiwan LLM invites collaboration and further innovation, ensuring that the linguistic diversity of Chinese speakers is embraced and well-served. The model, datasets, and further resources are made publicly available to foster ongoing research and development in this field.

翻译：在语言模型领域中，台湾地区使用的繁体中文所具有的细腻语言与文化复杂性长期受到忽视。本文提出台湾大语言模型，这是一款开创性的大型语言模型，专为繁体中文量身定制，并侧重于台湾地区使用的中文变体。通过利用全面的预训练语料库与指令微调数据集，我们开发的模型不仅理解繁体中文的复杂性，更融入了台湾的文化语境。台湾大语言模型作为同类首创，既在语言上准确无误，又能在文化层面与用户群体产生共鸣。评估结果表明，台湾大语言模型在理解与生成繁体中文文本方面表现卓越，超越了现有主要基于简体中文或英文训练的模型。该模型的开源发布将促进合作与进一步创新，确保中文使用者的语言多样性得到充分接纳与妥善服务。模型、数据集及其他相关资源均已公开提供，以推动该领域的持续研究与发展。