Gender bias in text corpora that are used for a variety of natural language processing (NLP) tasks, such as for training large language models (LLMs), can lead to the perpetuation and amplification of societal inequalities. This phenomenon is particularly pronounced in gendered languages like Spanish or French, where grammatical structures inherently encode gender, making the bias analysis more challenging. A first step in quantifying gender bias in text entails computing biases in gender representation, i.e., differences in the prevalence of words referring to males vs. females. Existing methods to measure gender representation bias in text corpora have mainly been proposed for English and do not generalize to gendered languages due to the intrinsic linguistic differences between English and gendered languages. This paper introduces a novel methodology that leverages the contextual understanding capabilities of LLMs to quantitatively measure gender representation bias in Spanish corpora. By utilizing LLMs to identify and classify gendered nouns and pronouns in relation to their reference to human entities, our approach provides a robust analysis of gender representation bias in gendered languages. We empirically validate our method on four widely-used benchmark datasets, uncovering significant gender prevalence disparities with a male-to-female ratio ranging from 4:1 to 6:1. These findings demonstrate the value of our methodology for bias quantification in gendered language corpora and suggest its application in NLP, contributing to the development of more equitable language technologies.

翻译：用于多种自然语言处理（NLP）任务（例如训练大型语言模型）的文本语料库中存在的性别偏差，可能导致社会不平等的延续和放大。这一现象在西班牙语或法语等性别化语言中尤为显著，因为其语法结构本身编码了性别信息，使得偏差分析更具挑战性。量化文本中性别偏差的第一步涉及计算性别表征偏差，即指代男性与女性的词汇在出现频率上的差异。现有测量文本语料库中性别表征偏差的方法主要针对英语设计，由于英语与性别化语言之间的内在语言差异，这些方法无法推广至性别化语言。本文提出一种新颖方法，利用大型语言模型的情境理解能力，定量测量西班牙语料库中的性别表征偏差。通过运用大型语言模型识别和分类与人类实体指称相关的性别化名词及代词，我们的方法为性别化语言中的性别表征偏差提供了稳健的分析。我们在四个广泛使用的基准数据集上对方法进行了实证验证，揭示了显著的性别分布差异，其男女比例范围在4:1至6:1之间。这些发现证明了本方法在性别化语言语料库偏差量化中的价值，并展示了其在自然语言处理领域的应用潜力，有助于开发更公平的语言技术。