The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social media. We first introduce the ViTHSD - a targeted hate speech detection dataset for Vietnamese Social Media Texts. The dataset contains 10K comments, each comment is labeled to specific targets with three levels: clean, offensive, and hate. There are 5 targets in the dataset, and each target is labeled with the corresponding level manually by humans with strict annotation guidelines. The inter-annotator agreement obtained from the dataset is 0.45 by Cohen's Kappa index, which is indicated as a moderate level. Then, we construct a baseline for this task by combining the Bi-GRU-LSTM-CNN with the pre-trained language model to leverage the power of text representation of BERTology. Finally, we suggest a methodology to integrate the baseline model for targeted hate speech detection into the online streaming system for practical application in preventing hateful and offensive content on social media.
翻译:社交网络的增长使得有害内容迅速传播。仇恨言论检测是一项有助于减少有害评论数量的任务。由于用户生成的仇恨言论具有多样性,除了检测仇恨言论外,还需对其进行解释。为此,我们提出了一种方法,用于构建从社交媒体在线流文本中进行目标仇恨言论检测的系统。首先,我们介绍了ViTHSD——一个面向越南社交媒体文本的目标仇恨言论检测数据集。该数据集包含10K条评论,每条评论针对特定目标标注为三个等级:清洁、冒犯和仇恨。数据集中有5个目标,每个目标均由人工根据严格的标注指南标注相应等级。通过Cohen's Kappa指数计算,数据集获得的标注者间一致性为0.45,属于中等水平。其次,我们通过结合Bi-GRU-LSTM-CNN与预训练语言模型,利用BERTology文本表示的能力,构建了该任务的基线模型。最后,我们提出了一种方法,将该基线模型集成到在线流系统中以进行目标仇恨言论检测,从而在实际应用中防止社交媒体上的仇恨和冒犯内容。