Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have been reported to encode worrying-levels of gender biases. Prior work has proposed debiasing methods that require human labelled examples, data augmentation and fine-tuning of LLMs, which are computationally costly. Moreover, one might not even have access to the model parameters for performing debiasing such as in the case of closed LLMs such as GPT-4. To address this challenge, we propose bias suppression that prevents biased generations of LLMs by simply providing textual preambles constructed from manually designed templates and real-world statistics, without accessing to model parameters. We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2. Moreover, we find that gender-neutral descriptions of gender-biased objects can also suppress their gender biases. Moreover, we show that bias suppression has acceptable adverse effect on downstream task performance with HellaSwag and COPA.

翻译：尽管大语言模型（LLMs）在各类自然语言处理任务中表现出色，但已有研究揭示其编码了令人担忧程度的性别偏见。现有去偏方法通常需要人工标注样本、数据增强及模型微调，计算成本高昂。更甚者，在GPT-4等封闭式LLMs场景中，研究者甚至无法访问模型参数进行去偏操作。针对这一挑战，我们提出无需访问模型参数的偏见抑制方法——仅通过构建基于人工设计模板与现实世界统计数据的文本前导语，即可阻止LLMs产生有偏生成内容。基于CrowsPairs数据集的实验表明，包含反事实陈述的文本前导语能有效抑制LLaMA2等英文LLMs中的性别偏见。此外，我们发现对性别偏见对象进行中性化描述同样可抑制其性别偏见。在HellaSwag与COPA数据集上的评估证实，该偏见抑制方法对下游任务性能的负面影响在可接受范围内。