Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research.While the genealogical ties between Creoles and a number of highly-resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data. In this work we present CreoleVal, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension, relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CreoleVal as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe.

翻译：克里奥尔语是一组未被充分探索且边缘化的语言群体，自然语言处理研究可用的资源极少。尽管克里奥尔语与若干资源丰富的语言之间存在谱系联系，这意味着迁移学习具有显著潜力，但由于标注数据的匮乏，这种潜力受到阻碍。本文提出了CreoleVal——一个涵盖8种不同自然语言处理任务、覆盖多达28种克里奥尔语的基准数据集集合；它整合了针对克里奥尔语的阅读理解、关系分类和机器翻译等新颖的开发数据集，同时为若干现有基准提供了实用入口。针对每个基准，我们在零样本设置下进行了基线实验，以进一步确定迁移学习对克里奥尔语的能力与局限性。最终，我们将CreoleVal视为推动自然语言处理与计算语言学领域对克里奥尔语研究的契机，总体而言，这是迈向全球更公平语言技术的一步。