Meenzerisch, the dialect spoken in the German city of Mainz, is also the traditional language of the Mainz carnival, a yearly celebration well known throughout Germany. However, Meenzerisch is on the verge of dying out-a fate it shares with many other German dialects. Natural language processing (NLP) has the potential to help with the preservation and revival efforts of languages and dialects. However, so far no NLP research has looked at Meenzerisch. This work presents the first research in the field of NLP that is explicitly focused on the dialect of Mainz. We introduce a digital dictionary-an NLP-ready dataset derived from an existing resource (Schramm, 1966)-to support researchers in modeling and benchmarking the language. It contains 2,351 words in the dialect paired with their meanings described in Standard German. We then use this dataset to answer the following research questions: (1) Can state-of-the-art large language models (LLMs) generate definitions for dialect words? (2) Can LLMs generate words in Meenzerisch, given their definitions? Our experiments show that LLMs can do neither: the best model for definitions reaches only 6.27% accuracy and the best word generation model's accuracy is 1.51%. We then conduct two additional experiments in order to see if accuracy is improved by few-shot learning and by extracting rules from the training set, which are then passed to the LLM. While those approaches are able to improve the results, accuracy remains below 10%. This highlights that additional resources and an intensification of research efforts focused on German dialects are desperately needed.
翻译:美因茨方言是德国美因茨市使用的方言,也是德国全国闻名的年度庆典——美因茨狂欢节的传统语言。然而,美因茨方言正濒临消亡,这也是许多其他德国方言的共同命运。自然语言处理(NLP)在语言和方言的保护与复兴工作中具有潜力。然而,迄今为止尚无NLP研究关注美因茨方言。本研究首次在NLP领域开展针对美因茨方言的专项研究。我们引入了一个数字化词典——一个源自现有资源(Schramm, 1966)的、可直接用于NLP的数据集,以支持研究者对该语言进行建模和基准测试。该数据集包含2,351个方言词汇及其对应的标准德语释义。随后,我们利用该数据集探究以下研究问题:(1)最先进的大语言模型(LLM)能否生成方言词汇的定义?(2)LLM能否根据给定定义生成美因茨方言词汇?实验结果表明,LLM在这两方面均表现不佳:最佳的定义生成模型准确率仅为6.27%,而最佳词汇生成模型的准确率仅为1.51%。为进一步探究准确率能否通过小样本学习以及从训练集中提取规则并输入LLM的方式得到提升,我们进行了两项补充实验。虽然这些方法能够改善结果,但准确率仍低于10%。这凸显出针对德国方言的额外资源和强化研究工作的迫切需求。