The predominance of English and Latin-based large language models (LLMs) has led to a notable deficit in native Arabic LLMs. This discrepancy is accentuated by the prevalent inclusion of English tokens in existing Arabic models, detracting from their efficacy in processing native Arabic's intricate morphology and syntax. Consequently, there is a theoretical and practical imperative for developing LLMs predominantly focused on Arabic linguistic elements. To address this gap, this paper proposes ArabianGPT, a series of transformer-based models within the ArabianLLM suite designed explicitly for Arabic. These models, including ArabianGPT-0.1B and ArabianGPT-0.3B, vary in size and complexity, aligning with the nuanced linguistic characteristics of Arabic. The AraNizer tokenizer, integral to these models, addresses the unique morphological aspects of Arabic script, ensuring more accurate text processing. Empirical results from fine-tuning the models on tasks like sentiment analysis and summarization demonstrate significant improvements. For sentiment analysis, the fine-tuned ArabianGPT-0.1B model achieved a remarkable accuracy of 95%, a substantial increase from the base model's 56%. Similarly, in summarization tasks, fine-tuned models showed enhanced F1 scores, indicating improved precision and recall in generating concise summaries. Comparative analysis of fine-tuned ArabianGPT models against their base versions across various benchmarks reveals nuanced differences in performance, with fine-tuning positively impacting specific tasks like question answering and summarization. These findings underscore the efficacy of fine-tuning in aligning ArabianGPT models more closely with specific NLP tasks, highlighting the potential of tailored transformer architectures in advancing Arabic NLP.

翻译：英语及拉丁语系大语言模型的统治地位导致原生阿拉伯语大语言模型的显著缺失。现有阿拉伯语模型中普遍包含英语词元，这进一步加剧了该问题，削弱了模型处理阿拉伯语复杂形态和句法的能力。因此，从理论和实践角度而言，开发以阿拉伯语语言元素为核心的大语言模型具有迫切性。为填补这一空白，本文提出ArabianGPT——ArabianLLM系列中专为阿拉伯语设计的Transformer模型。这些模型（包括ArabianGPT-0.1B和ArabianGPT-0.3B）在规模和复杂度上有所差异，以契合阿拉伯语精细化的语言特征。其中集成的AraNizer分词器针对阿拉伯文字独特的形态学特征进行优化，确保更精确的文本处理。在情感分析和文本摘要等任务上的微调实证结果表明模型性能显著提升：在情感分析任务中，微调后的ArabianGPT-0.1B模型准确率达到95%，较基础模型的56%实现大幅跃升；在摘要任务中，微调模型的F1分数得到增强，表明生成简洁摘要的精确率和召回率均有改善。通过将微调后的ArabianGPT模型与基础版本在多个基准测试中进行对比分析，可观察到性能的细致差异——微调对问答和摘要等特定任务产生积极影响。这些发现充分证明了微调在使ArabianGPT模型更贴合具体自然语言处理任务方面的有效性，彰显了定制化Transformer架构推动阿拉伯语自然语言处理发展的潜力。