Understanding commonsense knowledge is crucial in the field of Natural Language Processing (NLP). However, the presence of demographic terms in commonsense knowledge poses a potential risk of compromising the performance of NLP models. This study aims to investigate and propose methods for enhancing the performance and effectiveness of a commonsense polarization classifier by mitigating the influence of demographic terms. Three methods are introduced in this paper: (1) hierarchical generalization of demographic terms (2) threshold-based augmentation and (3) integration of hierarchical generalization and threshold-based augmentation methods (IHTA). The first method involves replacing demographic terms with more general ones based on a term hierarchy ontology, aiming to mitigate the influence of specific terms. To address the limited bias-related information, the second method measures the polarization of demographic terms by comparing the changes in the model's predictions when these terms are masked versus unmasked. This method augments commonsense sentences containing terms with high polarization values by replacing their predicates with synonyms generated by ChatGPT. The third method combines the two approaches, starting with threshold-based augmentation followed by hierarchical generalization. The experiments show that the first method increases the accuracy over the baseline by 2.33%, and the second one by 0.96% over standard augmentation methods. The IHTA techniques yielded an 8.82% and 9.96% higher accuracy than threshold-based and standard augmentation methods, respectively.
翻译:理解常识知识在自然语言处理(NLP)领域至关重要。然而,常识知识中人口统计术语的存在可能损害NLP模型的性能。本研究旨在通过减轻人口统计术语的影响,探索并提出增强常识极化分类器性能与有效性的方法。本文介绍了三种方法:(1)人口统计术语的层次泛化;(2)基于阈值的增强;(3)层次泛化与基于阈值增强方法的集成(IHTA)。第一种方法基于术语层次本体,将人口统计术语替换为更一般的术语,旨在减轻特定术语的影响。针对偏见相关信息有限的问题,第二种方法通过比较模型在掩码与未掩码人口统计术语时预测的变化,测量这些术语的极化程度。该方法通过使用ChatGPT生成的同义词替换谓语,对包含高极化值术语的常识句子进行增强。第三种方法结合了前两种方法,先进行基于阈值的增强,再进行层次泛化。实验表明,第一种方法相比基线准确率提高了2.33%,第二种方法相比标准增强方法提高了0.96%。IHTA技术相比基于阈值的增强方法和标准增强方法,准确率分别提高了8.82%和9.96%。