Bias in textual data can lead to skewed interpretations and outcomes when the data is used. These biases could perpetuate stereotypes, discrimination, or other forms of unfair treatment. An algorithm trained on biased data ends up making decisions that disproportionately impact a certain group of people. Therefore, it is crucial to detect and remove these biases to ensure the fair and ethical use of data. To this end, we develop a comprehensive and robust framework \textsc{Nbias} that consists of a data layer, corpus contruction, model development layer and an evaluation layer. The dataset is constructed by collecting diverse data from various fields, including social media, healthcare, and job hiring portals. As such, we applied a transformer-based token classification model that is able to identify bias words/ phrases through a unique named entity. In the assessment procedure, we incorporate a blend of quantitative and qualitative evaluations to gauge the effectiveness of our models. We achieve accuracy improvements ranging from 1% to 8% compared to baselines. We are also able to generate a robust understanding of the model functioning, capturing not only numerical data but also the quality and intricacies of its performance. The proposed approach is applicable to a variety of biases and contributes to the fair and ethical use of textual data.

翻译：文本数据中的偏见可能导致数据使用时产生曲解和偏差结果。这些偏见可能固化刻板印象、歧视或其他形式的不公平待遇。基于偏见数据训练的算法最终会做出对特定群体产生不成比例影响的决策。因此，检测并消除这些偏见对于确保数据使用的公平性和伦理性至关重要。为此，我们开发了一个包含数据层、语料库构建、模型开发层和评估层的综合性鲁棒框架 \textsc{Nbias}。数据集通过收集来自社交媒体、医疗健康和招聘门户等不同领域的多样化数据构建而成。在此基础上，我们应用基于Transformer的令牌分类模型，该模型能够通过独特的命名实体识别偏见的词语/短语。在评估过程中，我们融合了定量与定性评估方法，以衡量模型的有效性。与基线相比，我们实现了1%至8%的准确率提升。同时，我们能够生成对模型功能的鲁棒理解，不仅捕获数值数据，还体现了其性能的质量与复杂细节。所提出的方法适用于多种偏见类型，有助于实现文本数据的公平与伦理使用。