Within recent years, generative AI, such as large language models, has undergone rapid development. As these models become increasingly available to the public, concerns arise about perpetuating and amplifying harmful biases in applications. Gender stereotypes can be harmful and limiting for the individuals they target, whether they consist of misrepresentation or discrimination. Recognizing gender bias as a pervasive societal construct, this paper studies how to uncover and quantify the presence of gender biases in generative language models. In particular, we derive generative AI analogues of three well-known non-discrimination criteria from classification, namely independence, separation and sufficiency. To demonstrate these criteria in action, we design prompts for each of the criteria with a focus on occupational gender stereotype, specifically utilizing the medical test to introduce the ground truth in the generative AI context. Our results address the presence of occupational gender bias within such conversational language models.

翻译：近年来，生成式人工智能（如大型语言模型）发展迅速。随着这些模型日益向公众开放，人们开始担忧它们在应用中可能延续和放大有害偏见。性别刻板印象可能对所针对的个体造成伤害和限制，无论是表现为歪曲描述还是歧视。本文认识到性别偏见是一种普遍存在的社会建构，旨在研究如何揭示并量化生成语言模型中的性别偏见。具体而言，我们推导出分类领域中三种著名非歧视准则（即独立性、分离性和充分性）在生成式人工智能中的对应形式。为展示这些准则的实际应用，我们针对每个准则设计了提示语，重点关注职业性别刻板印象，并特别利用医学测试在生成式人工智能语境中引入真实标签。我们的研究结果揭示了此类对话语言模型中存在的职业性别偏见。