Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have been reported to encode worrying-levels of gender bias. Prior work has proposed debiasing methods that require human labelled examples, data augmentation and fine-tuning of the LLMs, which are computationally costly. Moreover, one might not even have access to the internal parameters for performing debiasing such as in the case of commercially available LLMs such as GPT-4. To address this challenge we propose bias suppression, a novel alternative to debiasing that does not require access to model parameters. We show that text-based preambles, generated from manually designed templates covering counterfactual statements, can accurately suppress gender biases in LLMs. Moreover, we find that descriptive sentences for occupations can further suppress gender biases. Interestingly, we find that bias suppression has a minimal adverse effect on downstream task performance, while effectively mitigating the gender biases.

翻译：尽管大型语言模型（LLMs）在广泛的自然语言处理任务中表现卓越，但已有研究报道其编码了令人担忧程度的性别偏见。先前提出的去偏方法需要人工标注示例、数据增强及对LLMs进行微调，这些方法计算成本高昂。此外，在处理如GPT-4等商业可用LLMs时，研究者甚至无法访问其内部参数以执行去偏操作。为解决这一挑战，我们提出偏差抑制这一新型替代方案，该方法无需访问模型参数。研究表明，基于人工设计模板（涵盖反事实陈述）生成的文本前缀可精准抑制LLMs中的性别偏见。进一步发现，职业描述性句子能强化对性别偏见的抑制效果。值得注意的是，偏差抑制在有效缓解性别偏见的同时，对下游任务性能的负面影响极小。