Recent advancements in Natural Language Processing (NLP) has led to the proliferation of large pretrained language models. These models have been shown to yield good performance, using in-context learning, even on unseen tasks and languages. They have also been exposed as commercial APIs as a form of language-model-as-a-service, with great adoption. However, their performance on African languages is largely unknown. We present a preliminary analysis of commercial large language models on two tasks (machine translation and text classification) across eight African languages, spanning different language families and geographical areas. Our results suggest that commercial language models produce below-par performance on African languages. We also find that they perform better on text classification than machine translation. In general, our findings present a call-to-action to ensure African languages are well represented in commercial large language models, given their growing popularity.

翻译：近期自然语言处理（NLP）领域的进展催生了大量预训练语言模型的涌现。研究表明，这类模型即使面对未见过的任务和语言，也能通过上下文学习展现良好性能，并通过语言模型即服务（language-model-as-a-service）的形式以商用API开放使用，获得了广泛采纳。然而，它们在非洲语言上的表现仍鲜为人知。我们针对覆盖不同语系与地理区域的八种非洲语言，对商用大语言模型在机器翻译与文本分类两项任务上开展了初步分析。结果表明，商用语言模型在非洲语言上的表现未达预期；同时，模型在文本分类任务中的表现优于机器翻译。总体而言，鉴于商用大语言模型的日益普及，我们的发现呼吁业界采取行动，确保非洲语言能在其中获得充分代表。