Hate detection has long been a challenging task for the NLP community. The task becomes complex in a code-mixed environment because the models must understand the context and the hate expressed through language alteration. Compared to the monolingual setup, we see very less work on code-mixed hate as large-scale annotated hate corpora are unavailable to make the study. To overcome this bottleneck, we propose using native language hate samples. We hypothesise that in the era of multilingual language models (MLMs), hate in code-mixed settings can be detected by majorly relying on the native language samples. Even though the NLP literature reports the effectiveness of MLMs on hate detection in many cross-lingual settings, their extensive evaluation in a code-mixed scenario is yet to be done. This paper attempts to fill this gap through rigorous empirical experiments. We considered the Hindi-English code-mixed setup as a case study as we have the linguistic expertise for the same. Some of the interesting observations we got are: (i) adding native hate samples in the code-mixed training set, even in small quantity, improved the performance of MLMs for code-mixed hate detection, (ii) MLMs trained with native samples alone observed to be detecting code-mixed hate to a large extent, (iii) The visualisation of attention scores revealed that, when native samples were included in training, MLMs could better focus on the hate emitting words in the code-mixed context, and (iv) finally, when hate is subjective or sarcastic, naively mixing native samples doesn't help much to detect code-mixed hate. We will release the data and code repository to reproduce the reported results.
翻译:仇恨检测长期以来一直是自然语言处理领域的一项挑战性任务。在代码混合环境中,该任务变得更为复杂,因为模型必须理解通过语言转换所表达的上下文和仇恨。与单语设置相比,由于缺乏大规模带标注的仇恨语料库来进行研究,针对代码混合仇恨的工作非常有限。为克服这一瓶颈,我们提出使用原生语言仇恨样本。我们假设在多语言语言模型时代,代码混合环境中的仇恨检测可以主要依赖原生语言样本实现。尽管自然语言处理文献报道了多语言语言模型在许多跨语言设置中仇恨检测的有效性,但它们在代码混合场景中的广泛评估仍有待进行。本文试图通过严谨的实验来填补这一空白。我们以印地语-英语代码混合设置为案例进行研究,因为我们具备相应的语言学专业知识。我们得到了一些有趣的发现:(i) 在代码混合训练集中加入原生仇恨样本,即使数量很少,也能提高多语言语言模型在代码混合仇恨检测中的性能;(ii) 仅用原生样本训练的多语言语言模型被观察到能在很大程度上检测代码混合仇恨;(iii) 注意力分数的可视化显示,当训练中包含原生样本时,多语言语言模型能更好地聚焦于代码混合上下文中传递仇恨的词汇;(iv) 最后,当仇恨表达具有主观性或讽刺性时,简单地混合原生样本对检测代码混合仇恨帮助不大。我们将发布数据和代码仓库以复现报告的结果。