Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural language processing. We analyse definitions of gender and its categories within social sciences and connect them to formal definitions of gender bias in NLP research. We survey lexica and datasets applied in research on gender bias and then compare and contrast approaches to detecting and mitigating gender bias. We find that research on gender bias suffers from four core limitations. 1) Most research treats gender as a binary variable neglecting its fluidity and continuity. 2) Most of the work has been conducted in monolingual setups for English or other high-resource languages. 3) Despite a myriad of papers on gender bias in NLP methods, we find that most of the newly developed algorithms do not test their models for bias and disregard possible ethical considerations of their work. 4) Finally, methodologies developed in this line of research are fundamentally flawed covering very limited definitions of gender bias and lacking evaluation baselines and pipelines. We suggest recommendations towards overcoming these limitations as a guide for future research.

翻译：本文对304篇关于自然语言处理中的性别偏见的论文进行了调查。我们分析了社会科学中的性别定义及其类别,并将这些定义与全国语言方案研究中的性别偏见的正式定义联系起来。我们调查了性别偏见研究中应用的词汇和数据集,然后比较和比较了发现和减轻性别偏见的方法。我们发现,关于性别偏见的研究有四个核心限制:(1) 多数研究将性别偏见视为一个二进制变量,忽视其流动性和连续性。(2) 大部分工作都是在英语或其他高资源语言的单语集中进行的。(3) 尽管在国家语言方案方法中有大量关于性别偏见的文件,但我们发现,大多数新开发的算法没有测试其偏见模式,也没有忽视其工作中可能的道德考虑。(4) 最后,我们发现,在这种研究中制定的方法存在根本的缺陷,包括性别偏见定义非常有限,缺乏评价基线和管道。我们建议克服这些局限性,作为未来研究的指南。