Over the last few years, Contextualized Pre-trained Neural Language Models, such as BERT, GPT, have shown significant gains in various NLP tasks. To enhance the robustness of existing pre-trained models, one way is adversarial examples generation and evaluation for conducting data augmentation or adversarial learning. In the meanwhile, gender bias embedded in the models seems to be a serious problem in practical applications. Many researches have covered the gender bias produced by word-level information(e.g. gender-stereotypical occupations), while few researchers have investigated the sentence-level cases and implicit cases. In this paper, we proposed a method to automatically generate implicit gender bias samples at sentence-level and a metric to measure gender bias. Samples generated by our method will be evaluated in terms of accuracy. The metric will be used to guide the generation of examples from Pre-trained models. Therefore, those examples could be used to impose attacks on Pre-trained Models. Finally, we discussed the evaluation efficacy of our generated examples on reducing gender bias for future research.

翻译：在过去几年里,诸如BERT、GPT等背景化的接受过训练的神经语言模型在各种国家语言方案任务中取得了显著进展。为了增强现有接受训练的模型的稳健性,一种方法是为数据扩充或对抗性学习生成对抗性实例和评价数据扩充或对抗性学习;与此同时,模型中嵌入的性别偏见似乎在实际应用中是一个严重问题。许多研究涵盖了字级信息产生的性别偏见(例如性别-陈规定型职业),而研究人员很少调查判决级案例和隐含案例。在本文件中,我们提出了一个在判决一级自动生成隐含性别偏见样本的方法和衡量性别偏见的衡量标准。我们的方法将用准确性来评估通过方法生成的样本。该标准将用来指导从受过训练的模式中生成范例。因此,这些范例可用于对受过训练的模型进行攻击。最后,我们讨论了我们所生成的减少性别偏见的实例在将来的研究中的评价效果。