Language models are trained mostly on Web data, which often contains social stereotypes and biases that the models can inherit. This has potentially negative consequences, as models can amplify these biases in downstream tasks or applications. However, prior research has primarily focused on the English language, especially in the context of gender bias. In particular, grammatically gender-neutral languages such as Turkish are underexplored despite representing different linguistic properties to language models with possibly different effects on biases. In this paper, we fill this research gap and investigate the significance of gender bias in Turkish language models. We build upon existing bias evaluation frameworks and extend them to the Turkish language by translating existing English tests and creating new ones designed to measure gender bias in the context of T\"urkiye. Specifically, we also evaluate Turkish language models for their embedded ethnic bias toward Kurdish people. Based on the experimental results, we attribute possible biases to different model characteristics such as the model size, their multilingualism, and the training corpora. We make the Turkish gender bias dataset publicly available.

翻译：语言模型主要基于网络数据进行训练，而这些数据往往包含社会刻板印象和偏见，模型可能继承这些特征。这会产生潜在的负面后果，因为模型可能在下游任务或应用中将此类偏见放大。然而，先前的研究主要聚焦于英语语境，尤其是在性别偏见方面。尤其值得关注的是，土耳其语等语法性别中性的语言虽具有不同的语言学特性，可能对语言模型产生不同的偏见效应，却尚未得到充分探索。本文旨在填补这一研究空白，考察土耳其语言模型中性别偏见的显著性。我们基于现有的偏见评估框架，通过翻译现有英语测试集并设计针对土耳其语区背景的性别偏见测评新任务，将其扩展至土耳其语环境。具体而言，我们还评估了土耳其语言模型对库尔德人群体的隐含种族偏见。基于实验结果，我们将可能的偏见归因于模型规模、多语言性及训练语料库等不同模型特征。我们将公开土耳其语性别偏见数据集。