We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for the countries of Brazil, Germany, India and Kenya, to aid training and interpretability of models. We demonstrate how our lexicon can be used to interpret model predictions, showing that models developed to classify extreme speech rely heavily on target words when making predictions. Further, we propose a method to aid shot selection for training in low-resource settings via HATELEXICON. In few-shot learning, the selection of shots is of paramount importance to model performance. In our work, we simulate a few-shot setting for German and Hindi, using HASOC data for training and the Multilingual HateCheck (MHC) as a benchmark. We show that selecting shots based on our lexicon leads to models performing better on MHC than models trained on shots sampled randomly. Thus, when given only a few training examples, using our lexicon to select shots containing more sociocultural information leads to better few-shot performance.
翻译:摘要:我们引入HATELEXICON——一个针对巴西、德国、印度和肯尼亚四国的侮辱性词汇及仇恨言论目标词词典——以辅助模型的训练与可解释性分析。我们展示了如何使用该词典解释模型预测,表明用于极端言论分类的模型在做出预测时高度依赖目标词汇。此外,我们提出一种方法,通过HATELEXICON帮助低资源场景下的训练样本选择。在少样本学习中,样本选择对模型性能至关重要。我们的工作基于HASOC数据训练,以多语言仇恨检测基准(Multilingual HateCheck,MHC)作为评估标准,模拟了德语和印地语的少样本场景。实验表明,基于词典选择样本训练的模型在MHC上的表现优于随机采样样本训练的模型。因此,当仅有少量训练示例时,利用词典选取包含更多社会文化信息的样本能显著提升少样本学习效果。