Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this paper, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly-available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

翻译：大语言模型的快速发展使其能够处理、理解和生成类人文本，并日益融入触及社会领域的系统中。尽管取得了成功，这些模型仍可能学习、延续并放大有害的社会偏见。本文对面向大语言模型的偏见评估与缓解技术进行了全面综述。我们首先整合、形式化并扩展了自然语言处理中的社会偏见与公平性概念，定义了不同维度的危害，并引入若干必要条件以实现大语言模型公平性的可操作化。随后，我们通过提出三种直观的分类体系（两种针对偏见评估，分别聚焦于评估指标与数据集；一种针对偏见缓解）对现有文献进行统一梳理。第一个评估指标分类体系厘清了指标与评估数据集之间的关系，并按其在模型中的不同作用层级（嵌入层、概率层与生成文本层）组织指标。第二个评估数据集分类体系根据结构将其分为反事实输入或提示，并识别了目标危害与社会群体；同时，我们整合了公开可用的数据集以便于访问。第三个偏见缓解技术分类体系根据干预阶段（预处理、训练中、处理中与后处理）对方法进行归类，并细分子类别以揭示研究趋势。最后，我们指出了未来工作的开放问题与挑战。通过综合近年来广泛的研究成果，本文旨在提供一份清晰的现有文献指南，帮助研究人员与实践者更深入地理解并预防大语言模型中偏见的传播。