The fairness of Natural Language Processing (NLP) models has emerged as a crucial concern. Information theory indicates that to achieve fairness, a model should not be able to predict sensitive variables, such as gender, ethnicity, and age. However, information related to these variables often appears implicitly in language, posing a challenge in identifying and mitigating biases effectively. To tackle this issue, we present a novel approach that operates at the embedding level of an NLP model, independent of the specific architecture. Our method leverages insights from recent advances in XAI techniques and employs an embedding transformation to eliminate implicit information from a selected variable. By directly manipulating the embeddings in the final layer, our approach enables a seamless integration into existing models without requiring significant modifications or retraining. In evaluation, we show that the proposed post-hoc approach significantly reduces gender-related associations in NLP models while preserving the overall performance and functionality of the models. An implementation of our method is available: https://github.com/fanny-jourdan/TaCo

翻译：自然语言处理模型的公平性已成为一项关键议题。信息论指出，为实现公平性，模型不应能预测诸如性别、民族和年龄等敏感变量。然而，与这些变量相关的信息常隐含在语言中，给有效识别和缓解偏见带来挑战。针对这一问题，我们提出了一种新颖方法，该方法在自然语言处理模型的嵌入层上运行，独立于具体架构。我们的方法借鉴了近期可解释人工智能技术的进展，并采用嵌入变换来消除所选变量中的隐含信息。通过直接操作最终层的嵌入，我们的方法能在无需大幅修改或重新训练的情况下，无缝集成至现有模型。评估中，我们展示了所提出的后处理方法能显著降低自然语言处理模型中与性别相关的关联，同时保持模型的整体性能和功能。本方法的实现见：https://github.com/fanny-jourdan/TaCo