This paper presents a pipeline for mitigating gender bias in large language models (LLMs) used in medical literature by neutralizing gendered occupational pronouns. A dataset of 379,000 PubMed abstracts from 1965-1980 was processed to identify and modify pronouns tied to professions. We developed a BERT-based model, ``Modern Occupational Bias Elimination with Refined Training,'' or ``MOBERT,'' trained on these neutralized abstracts, and compared its performance with ``1965Bert,'' trained on the original dataset. MOBERT achieved a 70\% inclusive replacement rate, while 1965Bert reached only 4\%. A further analysis of MOBERT revealed that pronoun replacement accuracy correlated with the frequency of occupational terms in the training data. We propose expanding the dataset and refining the pipeline to improve performance and ensure more equitable language modeling in medical applications.

翻译：本文提出了一种流程，旨在通过中性化处理与职业相关的性别代词，从而减轻医疗文献中使用的大语言模型（LLMs）中的性别偏见。我们处理了一个包含1965年至1980年间379,000篇PubMed摘要的数据集，以识别并修改与职业相关的代词。我们开发了一个基于BERT的模型，名为“现代职业偏见消除与精炼训练”（Modern Occupational Bias Elimination with Refined Training），简称“MOBERT”，该模型在此中性化摘要数据集上训练，并将其性能与在原始数据集上训练的“1965Bert”模型进行了比较。MOBERT实现了70%的包容性替换率，而1965Bert仅达到4%。对MOBERT的进一步分析表明，代词替换的准确性与训练数据中职业术语的出现频率相关。我们建议扩展数据集并优化流程，以提升性能，并确保在医疗应用中进行更公平的语言建模。