The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social media. We first introduce the ViTHSD - a targeted hate speech detection dataset for Vietnamese Social Media Texts. The dataset contains 10K comments, each comment is labeled to specific targets with three levels: clean, offensive, and hate. There are 5 targets in the dataset, and each target is labeled with the corresponding level manually by humans with strict annotation guidelines. The inter-annotator agreement obtained from the dataset is 0.45 by Cohen's Kappa index, which is indicated as a moderate level. Then, we construct a baseline for this task by combining the Bi-GRU-LSTM-CNN with the pre-trained language model to leverage the power of text representation of BERTology. Finally, we suggest a methodology to integrate the baseline model for targeted hate speech detection into the online streaming system for practical application in preventing hateful and offensive content on social media.
翻译:社交网络的增长使得有害内容迅速传播。仇恨言论检测是一项旨在减少有害评论数量的任务。鉴于用户生成的仇恨言论具有多样性,除检测外,对其加以解释也属必要。为此,我们提出一种方法论,用于构建面向社交媒体在线流文本的目标仇恨言论检测系统。我们首先介绍了ViTHSD——一个针对越南社交媒体文本的目标仇恨言论检测数据集。该数据集包含10K条评论,每条评论均按三个级别(无攻击性、冒犯性、仇恨性)标注至特定目标。数据集中包含5个目标,每个目标均由标注人员依据严格的标注准则手动标注相应级别。通过Cohen's Kappa指数计算,数据集的标注者间一致性为0.45,属于中等水平。随后,我们通过将Bi-GRU-LSTM-CNN与预训练语言模型相结合,构建了该任务的基线模型,以充分利用BERTology在文本表示方面的优势。最后,我们提出一种将目标仇恨言论检测基线模型集成到在线流处理系统的方法论,旨在实际应用于防范社交媒体上的仇恨与冒犯性内容。